InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusion
Authors: Yuanyi Wang
Zhaoyi Yan
Yiming Zhang
Qi Zhou
Yanggan Gu
Fei Wu
Hongxia Yang
Venue: NeurIPS 2025 poster ()
Keywords: Model Fusion
LLM
Primary area: deep_learning

Abstract: Recent advances in large language models (LLMs) have intensified efforts to fuse heterogeneous open-source models into a unified system that inherits their complementary strengths.  Existing logit-based fusion methods maintain inference efficiency but treat vocabulary dimensions independently, overlooking semantic dependencies encoded by cross-dimension interactions. These dependencies reflect how token types interact under a model's internal reasoning and are essential for aligning models with diverse generation behaviors. To explicitly model these dependencies, we propose \textbf{InfiGFusion}, the first structure-aware fusion framework with a novel \textit{Graph-on-Logits Distillation} (GLD) loss. Specifically, we retain the top-$k$ logits per output and aggregate their outer products across sequence positions to form a global co-activation graph, where nodes represent vocabulary channels and edges quantify their joint activations. To ensure scalability and efficiency, we design a sorting-based closed-form approximation that reduces the original $O(n^4)$ cost of Gromov-Wasserstein distance to $O(n \log n)$, with provable approximation guarantees. Experiments across multiple fusion settings show that GLD consistently improves fusion quality and stability. InfiGFusion outperforms SOTA models and fusion baselines across 11 benchmarks spanning reasoning, coding, and mathematics. It shows particular strength in complex reasoning tasks, with +35.6 improvement on Multistep Arithmetic and +37.06 on Causal Judgement over SFT, demonstrating superior multi-step and relational inference.

InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov- Wassv---sssrsttein for Model Fusion
Yuanyi Wang', Zhaoyi Yan', Yiming Zhang', Qi Zhou', Yanggan Gu',
Fei Wu², Hongxia Yangl.2 1.2x
The Hong Kong Polytechnic University, InfiX.ai, Zhejiang University edu. hk
yuan-yi.wangeconeee wang@connect. pow.tt...ooo...
Abstract
Recent advances in large language models (LLMs) have intensified efforts to fuse heterogeneous open-source models into unified system that inherits their complementary strengths. Existing logit-based fusion methods maintain inference efficiency but treat vocabulary dimensions independently overlooking semantic dependencies encoded by coss-dimension interactions. These dependencies reflect how token types interact under a model's internal reasoning and are essential for aligning models with diverse generation behaviors. To explicitly model these dependencies, we propose InfiGFusion, the first structure-aware fusion framework with novel Graph-on-Logits Distillation (GLD) loss. Specifically, retain the top-k logits per output and aggregate their outer products across sequence positions to form global co-activation graph, where nodes represent vocabulary channels and edges quantify their joint activations. To ensure scalability and efficiency, we design sorting- closed-fore approximation that reduces the original O(n²) cost of approximation guarantees. Experiments across multiple fusion settings show that GLD consistently improves fusion quality and stability. InfiGFusion outperforms SOTA models and fusion baselines across 11 benchmarks spanning reasoning, coding, and mathematics. It shows particular strength in complex reasoning tasks, with +35.6 improvement on Multistep Arithmetic and +37.06 on Causal Judgement over SFT, demonstrating superior multi-step and relational inference.
distance to O Iog with provable
Introduction
Recent advances in LLMs have sparked growing interest in aggregating diverse model capabilities to build stronger, more general-purposs systems. Existing collective approaches span a broad design space. Prompt-based techniques 2 dynamically chain compose pretrained LLMs with tools or APIs to expand functional scope. Multi-agent systems B employ multiple LLMs to collaborate or debate, often outperforming single-agent models on complex reasoning tasks. Model ensembling
5 6 aggregates outputs from different models to boost accuracy and robustness, while Mixture- architectures improve scalability by activating specialized subnetworks. Meanwhile, parameter-level merging models [10]
of-Experts (MoE)
11, 12 aim to consolidate multiple fine-tuned checkpoints into a single model. Each of these paradigms explores different trade-off' between performance, efficiency, and compatibility.
Among fusion strategies, model fusion via logit distillation 13 14 has emerged flexible and efficient paradigm. It enables a single pivot model to absorb knowledge from multiple source models-ofter with diverse domains, sizes, architectures wiithhhturrswwhhhuuttt increasing inference cost. A representative cross-tokening approach ULD [5], which introduces a universal logit- lostt-distillation
"Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

KL Loss & Wasserstein Distence
GLD Loss
Pivot Output
da
Pivot Logit Groph
Pivot Logit
d.
L
L Lea
2.
Source Legit
Source Logit Graph
Source Output
Independent Token-wise Alignment
Structure-awore Alignment
Groph Matching
Token-level Structure-aware Fusion. Given pivot and source logits of shape [T,3] (sequence length I,, vocab size 3), token-level methods (left) align dimensions independently, ignoring token interactions. GLD (right) aggregates outer products into [3, 3] co-activation graphs, capturing semantic dependencies via structure-awar graph alignment.
Figure 1:
loss for aligning models with different tokenizers. Unlike traditional distillation, which transfers in- termediate features outputs, these methods align only the output logits using token-level objectives such as KL divergence or Wasserstein distance [16]. However, token-level methods inherently treat each vocabulary dimension independently, overlooking semantic dependencies the co-activation patterns and relational structures among tokens that underlie a model's 's reasoning process. This limits their ability to align models with divergent generation behaviors, particularly for tasks requiring multi-step reasoning or fine-grained relational inference. Fig. Iilluutrrats this limitation. While token-level losses enforce dimension similarity, they fail capture the structured dependencies among tokens. InfiGFusion addresses this by modeling logits as graphs, aligning not just distributions but also the interactions that reflect a model's internal reasoning.
To address this challenge, we model semantic dependencies in the logit space using graph struc- tures. Rather than treating token logits independently, represent model outputs as feature-level graphs, where nodes correspond to token dimensions and edges capture their co-activation patterns across sequence positions. This representation reflects how model internally organizes semantic relationships, beyond surface-level token probabilities, to support coherent reasoning and inference. Constructing such graphs from high-dimensional logits is nontrivial: vocabulary sizes are large, and most logits are near-zero [17[8], making full pairwise modeling inefficient and noisy. To mitigate this, apply top-k sparsification to retain salient dimensions, followed by inner-product similarity to define edge weights. This yields interpretable graphs that faithfully reflect model reasoning behaviors.
Building on this, propose InfiGFusion, semantic structure aware fusion framework that distills knowledge from multiple source models by aligning their logit graphs. At its core is the Graph on- Logits Distillation (GLD) loss, which captures semantic dependencies by aligning structure-level information between source and pivot models. Specifically, we employ the Gromov- Wasserstein However, standard GW incurs (n*)) complexity, making it impractical for large vocabularies. To address this, we design a closed-form approximation based on node aggregation and sorting- based matching, reducing the cost O(n log n). We further provide theoretical guarantees on the approximation error, ensuring alignment fidelity.
(GW) distance match graph structures 19 20
We validate InfiGFusion across diverse fusion scenarios, demonstrating consistent improvements over state-of-the-ar methods. By explicitly modeling semantic dependencies through logit graph alignment, InfiGFusion excels in tasks involving multi-step reasoning, relational inference, and fine-gre fine-grained aligmment- scenaaros where token-level objectives fall short. Extensive experiments on 11 benchmarks, including reasoning, mathematics, and coding, confirm its effectiveness in producing more capable fusion models. Overall, our contributions can be summarized as follows:
InfiGFusion is the first structure-aww awarre fusion framework that models semantic dependencies via feature-level logit graphs and aligns them via a novel Craph-on-Logine Distillation loss.
We propose a novel approximation to Gromov- Wasserstein distance, reducing complexity from ((n2) to On log n.), with provable error bounds and rigorous theoretical analysis.
Experiments on 11 benchmarks show that InfiGFusion consistently outperforms baselines, with significant gains on complex reasoning tasks (+35.6 on Multistep Arithmetic, +37.06 Causal Judgement), demonstrating superior multi-step and relational inference.

---

2 Related Work
Collective LLM:
To aggregate the capabilities of specialized LLMs, several strategies have been pro-
posed. Prompt-based integration methods, such as Toolformer (1. ReAct [21], and HuggingGPT (22), guide LLMs to invoke tools or auxiliary models through natural language prompts. Mulii-agent
collaboration frameworks like ChatArena (23] and AgentBench 24 treat LLMs as interacting agents
that debate or coordinate to improve reasoning. Model ensembling aggregates outputs via voting (25) or reranking, but suffers from high inference cost. Mixture-of- Experts (MoE) models 7 [ activate sparse expert subsets per query, offering scalability but introducing routing complexity. Parameter merging methods [TO, 12 combine weights from fine-tuned models, but require architectural alignment. While these strategies explore different trade-offs between performance, efficiency, and flexibility, they generally assume the underlying models remain fixed. In contrast, we focus on knowledge fusion, where a pivot model learns from multiple sources during training.
Model Fusion via Knowledge Distillation: Traditional knowledge distillation (KD) transfers knowledge from a large teacher to a smaller student via features, logits, or output distributions [26] (27). primarily for compression or acceleration. In contrast, model fusion via KD aims to integrate multiple source models-offtee with varying architectures, scales, and vocabularies- into a single target of similar size. Recent methods [13 16 [4] fuse multiple LLMs by aligning token distributions using KL divergence or Wasserstein distance. These approaches typically require vocabulary alignment and assume semantic compatibility at the token level. However, they overlook semantic dependencies among logit dimensions- -ie., how token probabilities interact under internal reasoning, which are crucial for preserving logical structure. Our work addresses this gap by introducing structure-awe aware distillation objective that explicitly aligns semantic dependencies via graph alignment. Unlike prior methods, avoids explicit vocabulary alignment by modeling logits as graphs, enabling robust, vocabulary-agnostie fusion across heterogeneous models while preserving reasoning structures.
The Ceoomov-aaseesteer (GW) distance 228 20 relational optimal transport metric for aligning distributions based on internal structural similarity. It has been widely used in graph comparison 19 29 and structure-aware representation learning 50 31]
Gromov-Wasserste Wasserstein Distance:
especially when node correspondences are unknown. However, its standard formulation incurs O(n²) complexity due to fourth-order interactions, rendering impractical for large-scale logit graphs in LLMs. To mitigate this, prior works apply entropic regularization [20, (22), Sinkhorn- based solvers 19 33 or low-rank projections (34 While effective for small graphs, these methods lack closed-form solutions and suffer from convergence instability, making them less suited for distillation over vocabulary-sized graphs (n > 109). Our method takes non-iterative, sorting- approach: compresses each similarity matrix into node-level features and aligns them via sorted-based matching, reducing complexity to On log n1). Unlike prior approximations, it yields a diffcrentiable, training- friendly loss with provable approximation bound. To our knowledge, this is the first closed-fomm theoretically guaranteed GW approximation tailored for aligning logit graphs in LLM fusion.
3 Method
InfiGFusion fuses heterogeneous LLMs by distilling structured knowledge from multiple source models into a unified pivot model. Central to this process is the Graph-on-Logit. Distillation (GLD) loss, which models logit vectors as feature-level graphs and aligns their structures across models.
To capture semantic dependencies, we adopt Gromov- Wasserstein distance for structure-level align- ment between logit graphs. However, the standard GW incurs O(n") complexity, prohibitive for LLM-scale vocabularies (n > 10a). We propose sorting-based approximation that reduces this On log n)) by compressing graph structures into node-level features. Furthermore, we provide provable error bound of n=1 m-l (Proposition ensuring faithful and efficient graph alignment.
m
3.1 Overview of InfioFusion
InfiGFusion integrates knowledge from diverse source models (M,,..., Ms} into unified pivot model My, enhancing reasoning capabilities without increasing inference cost.
Given an instruction-response dataset DD Mo is trained to align with both human supervision and source model behaviors via two complementary objectives: (i) Universal Logit Distillation (UIDD): a coarse token-level alignment combining supervised fine-tuning (SFT) and a sorting based Wasserstein-
3

---

[1 Cart
I Token-wise
Structure lignment
Aligmment
Pivot
Topk
as Nodes
Graphs
Compression
Graph
0 000e0 c90e0
Parpaase
Sort & Topk
View as Nodes
Create Graphs
Compression
Graph Feat
Vi
Mi
Instraction
Sort TopK
V Veew Nodes
Create Graphs
Graph
Compression
Graph Feature Compression
Logits
Token-wise Aligmment
Structure A ligument
Node After Feat Comp.
WTIC, Ei #o - 60
GW(C, Dy Ei
Graph on Logits
InfiGFusion framework. Given instruction-response pairs, source and pivot models produce logits, sparsified into feature-level graphs capturing semantic dependencies. We align graphs via an efficient Cromov-Wassersee On n). The overall objective combines structure-aware distillation (GLD) with token-level distillation (ULD) and supervised signals (SFT) for robust fusion.
Figure 2:
approximation (GLD), reducing complexity from O(n*) to
1 approximation 15 relational dependencies in logit space. The overall loss is:
(ii) Graph-on-Logits Distillation (GLD): structure- objective aligning
LnniGFuuiion AGLD COLD. + AULD Σ CULD. + ASFTCSFT:
where ULD ensures distributional alignment and GLD captures fine-grained reasoning structures. For token-level ULD, adopt the linear-time Wasserstein-1 approximation [1]. which compare the logit distributions between pivot model / and source models s:
LUDD. WD f.)
Σ 10) 220
where ft denotes sorting. This provides an efficient coarse alignment signal, complementing GLD's structural matching.
3.2 Dynamic Sparse Graph Construction
To capture structural dependencies among logit dimensions while avoiding the cost of full d x modeling (d the vocabulary size for each model), we adopt a two-stage strategy: top-k sparsification followed by dynamic graph construction.
Top-k Logit Sparsification. Given logit tensor Z Bxdd extract the top-k logits for each position obtain T0P e Bxtdd, with d' & d. We then unify the top-k dimensions across tokens to obtain consistent d' dimennsiion representation per sample.
Dynamic Graph Construction. To model the interdependencies among logit dimensions, graph construction can generally follow two paradigms: (i) token-wise similarity across the sequence, or (ii) feature-wist similarity among logit dimensions. The former focuses on token interactions, while the latter captures intrinsic correlations between semantic dimensions. In this work, adopt the feature-wise approach, empirically yields better alignment performance and more effectively preserves structural information. Specifically, for each sample b, construct graph Gj, (V,E)

---

over the reduced logits Zn Riixd' where each node corresponds to selected logit dimension. The adjacency matrix C. E I xd' is defined via dot- dot-product similarity across the sequence:
Co(i,)) - Σ 2(11 2t(i,, C. = Z Z
This construction approximates cosine similarity after normalization and is applied to both pivot and source models every training step, resulting dynamic, data-dependent graphs that evolve with model predictions.
3.3 Efficient Approximation Algorithm
Directly applying Gromov- Wasserstin (GW) distance for structure-level alignment incurs O(n") complexity, which is infeasible for LLM-scale graphs. We propose an efficient approximation by summarizing graph structures into node-level features and aligning them via sorting-bass matching, reducing complexity to (n log n). This method maintains alignment fidelity with a provable error 1.. Full derivations provided in Appendix A
bound of 7 m-1 (Proposition
m²
Given graph similarity matrices C., C pdd''d' from source and pivot models, the squared GW distance is defined as:
D)
Σ |C(i,)) - D(k, Ta775 yEIl())))))) s.j.k.l
min
Despite its expressiveness, the GW objective requires O(n2m²) complexity due to its fourth-order tensor form, making it impractical for large logits (n, m are the vocabulary sizes). To address this, we derive closed-form approximation by compressing the graph structure into scalar features.
Graph Feature Compression. Our key insight is that the structure of each similarity matrix C' E RR and D E R can be summarized using scalar node-level features, where m are vocabulary size. Specifically, we compute degree-style features:
n fc(1) = n Σ C(.J), fb(k)
m
D(K,I),
(5)
m
which captures the overall importance connectivity of each logit dimension in the graph. This reduces the original d' x d' structure to two vectors in Rd
Separable Relaxation. Substituting these features into the original objective yields a simplified, separable form:
GW(C, - fD)
where symmetry is assumed across "/ik and T/r. This relaxation significantly lowers computational cost. As analyzed in Appendix and the following proposition, the error is theoretically bounded: Proposition (Approximation Error Bound). Let C E gux= and D mm be the similarity matrices derived from logit self-inner products and row-normalized such that E, Cij = and Σ, Dai 1. Then the absolute error between the exact and approximated GW distances satisfies:
GW(C, D) -GW(C, D)I
n
m
The proof is provided in Appendix A In practice, the size of m is more than 105. This bound guarantees the error decays as (1)nn), enabling scalable use in large-vocabulary settings.
Sorting-Based Matching, To avoid solving for the optimal transport plan 7 explicitly, we take a direct strategy like ULD [155 that adopts deterministic transport strategy based on sorting. Let Jb be the descending-sorte sorted versions of fc and fD. We construct a matching that aligns the i-th largest entry of fc to the i-th largest entry of fD:
GW(C, Σ 12(0) 1b00). == ==min(d,,d (d,
(8)
5

---

This effectively reduces the original optimization to an Ooglog sorting
sorting-based based assignment.
Stability Analysis. Beyond efficiency, approximation improves training stability. We compare the Lipschitz constants of common alignment losses and the proof is provided in Appendix C
Proposition 2 (Lipschitz Constants Comparison). We have the following relationship between the Lipschitz constants for the Gromov- Wasserstein, Wasserstein, and KI. losses:
Low = (
R
LuD = O(D)) < LKL LxL= O(e
for vocabulary size D and logit range R, which means the GW-guided training models have tighter generalization bounds.
The logit range R is usually smaller than 120 for 14B LLMs. For example, with vocabulary size
D 150, 000, is observed in the range of [40, 120] [35. This suggests GW-based alignment yields the lowest sensitivity to perturbations, contributing to improved generalization and stable training.
3.4 Unified GLD Loss
To support multi-source fusion, we extend the GLD loss into a unified formulation. For each source model M,, we compute structure-aware alignment loss with respect the pivot model Mo and aggregate all source contributions:
COLD Σ GW(C..Co).
(10)
where C. and Co denote the similarity matrices of the source and pivot graphs, respectively.
This unified objective enables efficient and scalable distillation from multiple source models, while preserving both semantic and relational structures across models.
Experiments
We evaluate InfiGFusion on large-scale L.M fusion, focusing on integrating models with diverse architectures and reasoning styles while maintaining inference efficiency. Our experiments verify that GLD enables faithful and effective fusion beyond token-level objectives. IntiGFusion outperforms state-of-the-arr fusion methods and strong open-source baselines, with significant improvements in multi-step reasoning and relational inference. Additional results are provided in the Appendix.
4.1 Experimental Setup
Types
General
Math
Code
Dataset
Infinity-Insmnet Numina Math-1.5 KodCode-VI
Datasets. We construct a novel multi-task train- ing dataset comprising 130k examples across gen- eral reasoning, mathematics, and coding. (i) The
Original Size
L.4M
268k
Filicred Size
52K
39K
39K
Table 1: Statistics of the novel datasets.
52K general data are sourced from Infinity-Instruct 39k pairs of math questions and corresponding answers. The questions are sourced from the
(36
(ii) Our mathematical dataset comprises
N TuminaM ath 1.3 1.52 dataset, while the answers distilled from the DeepSeck-R1-677 1B model by the AM team N "umminaaM oth 1.5 represents the second iteration the widely acclaimed NutinaMath| 337 dataset. It offers high-quality data for competition-leve mathematical problems across various domains, including Algebra, Geometry, Combinatorics, Calculus, Inequalities, Logic and Puzzles, and Number Theory. (iii) In the code generation domain, we utilized the KodCode-V V1- SFT-R1 dataset [8), which contains approximately 268k samples. Each sample was fed into our pivot model, which was prompted to generate 5 random responses per input. The generated outputs were then passed through sandbox-based evaluation system. For each sample, if at least one of the five responses failed, the sample was flagged for further consideration. From these flagged samples, we filtered and selected 39k high-quality examples for further distillation.
http://wnggiiiiee..waaeeeeeslsee -I- Distilledd1... 1.4M

---

Models
Math
Coding
General Reasoning
Instruct.
Text Reasoning
Model
GPU
Avg
GSMSK MATH
ThQA
MBPP HEval
BBH
ARC
MMLU
IFEval
DROP
HS
Size
Hour
Pivot Model
Phi-4
87.41
80.04
51.12
70.8
83.54
68.84
93.9
85.62
77.34
88.67
87.62
79.54
LOM
Source Models
Qwen2.5-Instruc
91.13
78.16
47.25
81.70
83.54
77.59
92.20
80.22
85.01
85.56
88.28
80.97
14R
-1.M
Mistral-Small
92.42
69.84
48.50
68.80
84.15
81.59
91.86
81.69
82.25
86.52
91.84
79.95
24B
1.6M
Qwen2. Sweder
74.18
38.88
85.40
90.90
75,40
89.49
75.08
74.70
84.34
79.83
77.94
-18M
SET
Piyot SFT
89.99
82.96
54,50
77.86
87.80
67.23
93.56 86.21
77.70
89.44
87.76
81.36
14B
120
Model Fusion via Distillation
MiniLogit
89.80
79.78
46.36
78.49
85.44
82.68
84.58
79.36
88.56
88.25
81.43
14B
220
F.
90.24
80.25
53.52
79.28
84.00
77.62 92.08
83.92
78.56
88.74
87.81
81.46
14B
225
FuseChat
91.21
77.52
51,88
81.80
84.15
83.37
93.56
84.23
78.90
89.23
87.42
82.12
14B
650
InfiFusion
90.07
80.94
54,62
79.63
84.72
80.94
94,24
85.81
76.02
89.27
87.91
82,20
14B
160
InfiGFusion
81.92
55.38
85.2
86.00
85.62 94.58
85.24
80.22
89.62
88.22
83.85
Table 2: Main results various benchmarks.
Reasoning Models
Qwen2.5-Instruct
Mistral
DeepSeck-RR Riii--RR-Dssiiii Disssiill-Qwen
Phi4
Phi4 (SFT)
InfiGFusion
Models Size
14B
24B
14B
14B
14B
14B
Abstract Algebra
73
85
86 (+4.07)
88 (+6.07)
Marketing International Law
90.17
92.31
89.32
92.74
92.74 (+0.0)
95.3 (+2.561)
81.82
81.82
82.64
91.74
90.91 (-0.83))
90.91 (-0.83.)
Moral Scenarios
71.96
68.6
73.41
75.75
74.75 (-1.01)
73.97 (-1.78J)
Virology
52.41
49.4
54.22
53.01
53.61 (+0.61)
54.22 (+1.21
Formal Logic
68.25
66.67
91.27
77.78
78.57 (+0.791)
74.6
Security Studies
77.96
78.78
78.37
77.14
(+2.457)
81.22 (+4.087)
logical_fallacies
85.28
84.66
85,28
87.73 (+12237)
87.73 (+1.23))
Ruin Names
82.8
76.8
39.6
88.8
88.0
88.4 (-0.41)
Tracking Objects
80.4
80.8
90.0 (-4.4L)
96.8 (+2.44)
Tracking Objects
79.2
99.2
82.8
9-1.
96.8 (+0.0)
Logical Deduction
97.6
98.8
83.2
96.4 (-2.0L)
97.6 (-0.8.)
Logical Deduction
80.4
85.6
92.4 (+6.81)
94 (+8.47)
Logical Deduction
68.8
62.8
83.2
88.4
88.8 (+0.41)
89.2 (+0.88)
Colored Objects
93.6
87.6
96.4
96.8 (+0.41)
96.4 (+0.0)
Multistep Arithmetic
96.4
93.2
81.6
62 (-2.0L)
99.6 (+35.61)
Dyck Languages
35.6
37.2
9.6
11.2
13.2 (+2.0T)
40.0 (+28.81)
Causal Judgement
43.85
68.98
45.35
32.99
35.83 (+2.847)
70.05 (+37.061)
Avg. 18 Tasks
75.53
77.37
73.29
77.43
77.94 (+0.511)
84.16 (16.7 73f)
Table Performance on complex reasoning tasks. These datasets involve multi-step logic, causal inference, event tracking, and semantic analysis. InfiGFusion consistently improves over SPT, especially in complex reasoning. More results (84 tasks) are provided in Appendix
Models and Baselines. fuse three source models: Qwen2.5-14B 339
Sowen2.5-Cdder 14B
and Mistral-24BB 40 into a Phi-4 [1 pivot model. Fusion baselines include Minil
42
InfiFusion 16] FuseLLM
[33 and FuseChat
Full configurations are provided in Table
Evaluation. We assess fusion performance on 11 benchmarks spanming reasoning (BBH 44 ARC-
455, MMLU (46)), mathematics (GSMSK (47). Math (48), TheoremQA (49)), coding (MBPP (50).
HumanEval [5I)), text reasoning (DROP (2). HellaSwag, (53)), and instruction following (IFE-
val 54
Additional training details and results are provided in Appendix
4.2 Main Results
Table summarizes the performance of InfiGFusion actoss eleven benchmarks, compared with the pivot model, source models, and representative model fusion methods.
Comparison with Pivot and Source Models. InfiGFusion consistently outperforms both the pivot model and source models in overall performance. Compared to the pivot model, achieves signifi- cant improvement (+2.53 avg), particularly on complex reasoning benchmarks such as BBH, ARC, and MMLU, where semantic richness and multi-step inference are crucial. For instance, InfiGFusion surpasses the pivot by +16.7 on BBH, demonstrating its ability to aggregate diverse reasoning styles beyond single-source capabilities. Compared to source models, InfiGFusion effectively integrates complementary strengths. While individual source models excel in isolated tasks (c.g., Mistral-S Small
7

---

on BBH, Qwen2.5-Cond 5-Coder on HEval), they struggle with generalization across domains. InfiGFusion surpasses these models with a more balanced performance, avoiding their trade-offs and consolidating knowledge into a unified pivot.
Comparison with Model Fusion Baselines. Against existing fusion methods (MiniL.ogit, LuseLLMM, FuseChat). InfiGFusion exhibits superior fusion quality. Notably, while other distillation methods provide moderate improvements over the pivot, they often suffer from degraded performance in high-dependency tasks (e.g., logical reasoning, multi-condition alignment). InfiGFusion consistently delivers better results on structurally challenging tasks, indicating its advantage in preserving inter- token dependencies during fusion. Furthermore, compared to simpler fusion objectives, the structure- aware GLD loss of InfiGFusion yields better alignment without sacrificing robustness.
Efficiency and Resource Usage. In terms of computational cost, InfiGFusion maintains competitive efficiency. Despite introducing structure-aware alignment, its GPU hours remain modest (195 GPU hours), substantially lower than multi-step fusion pipelines like FuseChat (650 GPU hours). Model size remains identical the pivot (14B), ensuring no additional inference overhead.
4.3 Complex Reasoning Performance
We evaluate InfiGFusion on 18 reasoning datasets spanning BBH and MMLU, covering multi-step logic, structural dependencies, and fine-grained semantic alignment, as shown in Table These tasks are known to challenge teken-level fusion methods due to their complex reasoning path. Comprehensive results can be found in Appendix
Improved Reasoning over SFT. Compared to Phi4- 14B after supervised fine-tuning (SFT), InfiG- Fusion delivers consistent gains, boosting the average accuracy from 77.94% to 83. 33.79%. Notable improvements are observed in tasks demanding complex relational reasoning: Multistep Arithmetic (+34.8%) and Causal Judgement (+34.39%). This demonstrates InfiGFusion's 's ability to enhance multi-hop inference and causal understanding by explicitly modeling semantic dependencies.
Structural Alignment Benefits. InfiGFusion excels in Tracking Shuffled Objects and Logical Deduction tasks, where aligning token relations is critical. For instance, it surpasses Phi4- 1hi4-14B SFT on Logical Deduction Five Objects (+1.6%) and Seven Objects (+0.4%), showcasing robust structure- levell fusion beyond token alignment.
Preserving Pivot Strength. InfiGFusion maintains competitive performance in knowledge-centric tasks like International Law and Ruin Names, matching or slightly improving upon SFT baselines. This indicates that InfiGFusion not only integrates external model knowledge but also preserves the pivot model's s original strengths. These results confirm that InfiGFusion's 's structure-aware fusion consistently enhances reasoning capabilities compared to token-level methods.
Model
Reasoning
Math
Coding
Avg
4.4 Ablation Study
Pivot Phi4
81.43
72.86
77.17
79.54
Qe
84.39 74.18 (+1.32)
86.1 (+8.63)
(-33.6))
Effect of Source Model Diversity. Table eval- evall uates the effect of fusing different source models. Individually. Qwen-Coder (Qc). Iwwnn-Inttrcct (Qi), and Mistral (M) each provide moderate gains, reflecting their nomain-specifte strengths. Notably, combinations such as Qi-M further am- plify reasoning (+3.57), math (+3.06), and cod- ing (+8.22) performance, demonstrating that complementary inductive biases from different models enhance the pivot model. InfiGFusion achieves the best results (+2.53 avg), confirming that our framework effectively consolidates diverse reasoning behaviors into a unified model.
Qi
86.03 (+4.60) 74.08 (+1.22) 84.89 (+7.72) K3.52 (+3.98)
M
84,50 (+3,07) 75.77 (+2.91) 85,8 (+H.63) 83.62 (-4.08)
Q--
84.89 (+3.46) 74.18 (+1.32) 86.09 (+8.92) 83.54 (-4.00)
Qi-M
85.00 (+3.57)) 75.92 (+3.06) 85.39 83.72 (-4.18)
InfiGFusion 87.06 (45.63) 74.74 (+1.88) (48.44) 83.85 (-2.53)
Table 4: Ablation study on model diversity. Qc, Qi, and M mean Coder, Qwen-Instruet, and Mistral.
Effect of Loss Components. Table 5 analyzes our loss design. Removing ULD (w/o CLD) leads to a significant drop (-1.52 avg), particularly in grained reasoning (-1.8), indicating that coarse coarse-grained token-level alignment is crucial for providing a reliable starting point. ULD narrows large dis- tribution mismatches, effectively preparing the model for the finer-grained structural alignment performed by GL.D, Excluding GLD (w/o GI.D), on the other hand, yields a smaller overall decline (-0.69 avg), but reasoning performance still drops notably (-1.36). This highlights that while ULD han-
Component GLD ULD
Reasoning
Math
Coding
Avg
87.06
74.74
83.85
ULD
85.26 (-1.) 73.28 (-146) 85.28 (-0.32) 82.33 (-1,52)
wio GLD
95.70 36) 75.21 (40.47) 85. (-0.42) 83.16 (-0.69)
Table 5: Ablation study on loss components.
8

---

dles first-order token matching, it lacks the capacity to capture higher-order semantic dependencies. When combined, ULD and GLD form a synergistic alignment mechanism: ULD aligns distributions coarsely and stabilizes optimization, while GL.D refines alignment by enforcing consistency in the relational structure of logits. The empirical results validate this design, with the full loss achieving the best performance. All ablation studies are tested in the Top-k = 10 setting.
4.5 Effect of Top-k
InfiGFusion sparsifies logits by retaining top-k token dimen- sions before graph construction, selecting the most salient indices per sequence position. This inductive bias suppresses noisy activations and emphasizes meaningful token depen- dencies, serving as the foundation for graph- basph seman- tic alignment. We evaluate Top-k {5, 10, 15, 20, 25, 30} and report the results Fig. Increasing & from 5 to 10 brings notable gains across all tasks (Avg +0.54), as larger captures richer token interactions essential for semantic graph construction. However, further increasing & beyond 10 yields diminishing returns and even slight degradations, particularly in reasoning (-0.49 from Top10 to Top30). This suggests that excessive low-confidence tokens introduce spu- rious edges, diluting graph discriminability. The observed trend aligns with the heavy-tailed nature of LLM logits [17]. where only small subset of token dimensions are informative. Overall, Top-10 10 achieves the best
Reasoning
Math
Coding
Avg
82
80
78
76
74
10
Top-k
Figure 3: Top-k analysis.
trade-off, balancing informative graph structure and computational efficiency.
4.6
Case study
To illustrate InfiGFusion's 's strengths beyond token-level alignment, we analyze representative cases, comparing outputs from standard SFT-tuned model (Phi4) and our InfiGFusion model. The selected examples highlight InfiGFusion's superior reasoning in multi-step causality and relational inference.
Case 1: Frank T. Shooting Incident. While both models predict "No," InfiGFusion performs deeper step-by-step causality analysis. It explicitly identi- fies the causal chain's disptiionnddsiiuuuisiinnn between "intent," "misfire." and "accidental re- sult" showcainnn robust step causality dis- ambignation capabilities.
Cose 1: Frank I. Shooting Incident
Question: Frank intending shoot neighbor, accidentally misfires due inexpperience, hitting rock, which couses the bullet to ricochet and injure the neighbor. Did Fronk "intentionally" hit the neighbor?
Phi4 (SFT)
InfiēFusion
No. Frank's hand slipped the shot missed, so was accidental.
No. Although Frank Initially Intended
shoot neighbor. the unintended ricochet broke the cousal link between
intention
and the actual injury. The injury was
result of an accident, not direct intent.
Case 2: Wallace's Dual Cause of Death. Unlike Phi4's surface-level judgment, InfiGFusion reasons through the latent causal structure, identifying "or- ganized crime" as a distal cause. This reflects its strength relational and indirect causality infer-
Case 2: Wallace's 5 Dual Couse of Death
Question: Crime boss Wallace targeted sinultaneously by poisoring and car bomb. The poison is fatal after hour, but Walloce dies instantly the explosioon. Did his involvement in organized crime "couse" his death?
Phi4 (SFT)
InfiéFusion
ence, where it effectively models complex event chains and upstream dependencies.
immediate
Yes. While the explosion the direct
attempts mo tivated by Wallace's criminal ties.
cause, both
couse
unrelated to the
the explesion,
gong involvement set in motion the chain of levents leading to his death, establishing on indirect cousal link.
poisoning.
These cases demonstrate that InfiGFusion excels at: (i) Multi-step inference: Decomposing complex causal chains into interpretable reasoning steps. (ii)
Figure 4: Case study.
Relational causality reasoning: Capturing indirect and upstream causal factors often missed by SFT models. (iii) Fine- tine-grained d disambiguation: Distinguishing intent, action, and outcome with structured alignment of reasoning behaviors.
4.7 Analysis of the GW approximation
In addition to the main results, we also conducted some experiments to analyze our proposed Gromov- Wasserstein approximation. It compares the quality of GW estimation in a simple simulated scenario. For example, by obtaining samples from two Gaussian distributions, and comparing the value of exact GW, sinkhorn- basdd approximation, and our proposed approximation. While our experiments

---

Fxact&&&Sikhoom GW
Approx GW
Approx RE (+sid)
Time Exact (8)
Time Sink (8)
Time Approx (8)
50
0.4815 + 0.0452
0.1978 = 0.0207
0.5883 = 0.0242
0.0171 = 0.0207
0.0027 = 0.0005
0.0004 = 0.0004
100
0.3247 +: 0.0297
0.1261 + 0.0217
0.6048 = 0.0907
0.0072 + 0.0008
0.0070 + 0.0005
0.0001 + 0.0000
200
0.2033 +: 0.0088
0,0973 0.0125
0.5230 =.30000000 0.0426
0.1390 = 0.0377
0.0983 + 0.0033
0.0359
0.0305
0.1131 + 0.0050
0.0587 0.0143
0.1122
0.4405 0.1337
0.4815 + 0.0904
0.0353
1000
0.0764 +: 0.0011
0.0387 + 0.0133
0,4921 0.11785
2.3631 + 0.4981
2.2456 + 0.4085
0.1238 0.0399
1500
0.0621 + 0.0028
0,0308 0.0016
0.5031 0.0398
12.7151 4.1182
9.1998 + 2.1952
0.0982 0.0015
2000
0.0496 + 0.0028
0,0294 + 0.0058
0.3995 + 0.1453
20.1236 5.0250
19.8941 8.1827
0.0411 + 0.0424
2500
0.0455 + 0.0008
0.0305 + 0.0075
0.3285 = 0.1712
20.9198 = 5.6782
30.1367 = 3,1038
0.0026 + 0.0014
3000
0.0404 +: 0.0002
0.0324 + 0.0141
0.3941 = 0.0767
52.8898 9.2735
53.5049 + 12.4434
0.2408 0.2562
Table 6: GW estimation on synthetic Gaussians. In our runs, Exact GW and Sinkhorn GW are
numerically indistinguishabl,
report them in a single column labeled
GW.
demonstrate that the approximation is suitable as a training objective, it would be interesting to also understand its behavior as GW estimator.
Experiment setup: For each n E (50, 100, 200, 500, 1000, 1500, 2000, 2500, 3000], we sam- ple points from (N0,,) I) and (N1, T), build their and compute: (i) Exact GW and Sinkhorn GW via ot. groooooov.ooo wwtpsoowwwssesseeeeeeeee (.... loss=' "aassssqurr__eee e_losss); (ii) Approx GW via our sorting-based closed-form. Each configuration is repeated 3x to report Ltdd. Approx RE quantifies the approximation accuracy relative to the ICW wome -Waas
n Euclidean distance matrices,
exact GW: RE
(mean over 5 seeds). In this symmetric Gaussian setting, Exact and Sinkhorn GW coincide to reported precision (even with 10-1), evidencing the high accuracy of the Sinkhorn estimator here.
GWessel
Results and analysis: Each setting is repeated 5 times with different seeds; we report ttdn Across all sizes, Sinkhorn GW matches Exact GW to our reported precision (RE 0), hence we merge them into single column in Table Our Approx GW exhibits decreasing relative error as grows (from R 0.59 at n = 50 down to Rs 0.39 at n = 3000), consistent with the O(1/n) behavior. In terms of runtime, Exact/Sinkhorn scale poorly and rapidly become impractical beyond a few hundred points (already seconds at n = 1000 and tens of seconds by n 2 1500 on our machine), whereas Approx GW remains below 0.3 even at n 30000, demonstrating its On log n) scalability.
5 Conclusion
In this work, we propose InfiGFusion, a structure-aware fusion framework that explicitly models semantic dependencies among logits through novel Graph-on-LLoiit Distillation (GLD) loss. By leveraging logit graph representations and an efficient Gromov-Wasserse approximation, InfiGFusion enables the integration of heterogencous LLMs without increasing inference cost. Extensive experiments on 11 benchmarks demonstrate that InfiGFusion consistently outperforms sfate-of-the-arr fusion methods and baselines. In particular, achieves notable gains in multi-step stelti-seeppoa and relational reasoning tasks, such as +35.6 on Multistep Arithmetic and +37.06 on Causal Judgement over SFT models. It highlights the importance of structure- -prettrrii alignment for effective model fusion in complex reasoning tasks, which is a promising direction for advancing collective I.I.M
Limitations
While InfiGFusion demonstrates clear advantages in tasks involving multi-step reasoning, causal inference, and relational alignment, its performance shows limitations in scenarios dominated by literal token matching or factual recall. For tasks where the correct answer is encoded by a few dominant logit dimensions (e.g., factoid QA), the added structural alignment of GLD brings marginal benefits and may even introduce noise from irrelevant dependencies. Moreover, when source models present conflicting factual knowledge, InfiGFusion focuses on preserving relational consistency but lacks explicit mechanisms for factual conflict resolution, sometimes amplifying semantic ambiguities. These limitations reflect inherent trade-off's in structure-aware fusion and motivate future work on adaptive dependency modeling and source reliability estimation.
Our model is available at http://huggnnngfaaee.o/iiiiaaaia
14B
10

---

References
[1] Timo Schick, Jane Awivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zet, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539 68551, 2023.
[2] Hui Yang, Sifu Yue, and Yunzhong He. Auto-gpt for online decision making: Benchmarks and additional opinions. arXiv preprint arXiv:2306.02224, 2023.
[3] Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Zou. Mixture-of-agente enhances large language model capabilities. arXiv preprint S4iv:2400.046999 2024.
[4] Yann Dubois, Balázs Galambosi, Percy Liang, and Tatsumori B Hashimoto. alpacaeval: A simple way debias automatic evaluators. arXiv preprint arXiv:2404.0444 2024.
[5] Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. Lim-blender: Ensembling large language models with pairwise ranking and generative fusion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14165-14178. 2023.
[6] Chenhao Fang, Xiaohan Li, Zezhong Fan, Jianpeng Xu, Kaushiki Nag, Evren Korpeoglu, Sushant Kumar, and Kannan Achan. Lim-ensemble: Optimal large language model ensemble method for e-commerce product attribute value extraction. In Proceedings of the 47th Interna- lional ACM SIGIR Conference Research and Development in Information Retrieval, pages 2914,2914, 2024.
[7] Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, and Neil Houlsby. Sparse upcycling: Training mixture-of-experts from dense checkpoints. In The Eleventh International Conference on Learning Representations.
[8] Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, al. Glam: Efficient scaling of language models with mixture-offexppets In International conference on machine learning. pages 5547-5569. PMLR, 2022.
[9] Zixiang Chen, Yihe Deng, Yue Wu, Quanquan Gu, and Yuanzhi Li. Towards understanding the mixture-of-experts layer in deep learning. Advances in neural information processing systems, 0123009-230066 2022.
[10] Prateck Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. Ties-mergin Resolving interference when merging models. Advances in Neural Information Processing Systems, 00666099971155 2023.
[11] Michael S Matena and Colin A Raffel. Merging models with fisher-weighted averaging. Advances in Neural Information Processing Systems, 35:17703-17716, 2022.
[12] Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Lontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International conference on machine learning, pages 23965-23999 PMLR, 2022.
[13] Fanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan, Wei Bi, and Shuming Shi. Knowl- edge fusion of large language models. In The Twelfih International Conference on Learning Representations.
[14] Tianyuan Shi, Fanqi Wan, Canbin Huang, Xiaojun Quan, Chenliang Li, Ming Yan, and Zhang. Profuser; Progressive fusion of large language models, arXiv preprint arXiv:2408. 04998, 2024.
11

---

[15] Nicolas Boizard, Kevin El Haddad, CELINE HUDELOT, and Pierre Colombo. Towards cross- tokenizer distillation: the universal logit distillation loss for 1lms. Transactions on Machine Learning Research.
[16] Zhaoyi Yan, Yiming Zhang, Baoyi He, Yuhao Fu, Qi Zhou, Zhijie Sang, Chunlin Ji, Shengyu Zhang, Fei Wu, and Hongxia Yang. Infifusion: A unified framework for enhanced cross-model reasoning via Ilm fusion. arXiv preprint arXiv:2501.00 0.2995, 2025.
[17] Shengyao Zhuang, Honglei Zhuang, Bevan Koopman, and Guido Zuccon. setwise approach for effeettive and highly efficient zero-shot ranking with large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 38 47, 2024.
[18] Chris Lu, Samuel Holt, Claudio Fanconi, Alex Chan, Jakob Foerster, Mihaela van der Schaar, and Robert Lange. Discovering preference optimization algorithms with and for large language models. Advances in Neural Information Processing Systems, 0188866288877777 2024.
[19] Hongteng Xu, Dixin Luo, and Lawrence Carin. Scalable gromov-wasserseinn learning for graph partitioning and matching. Advances in neural information processing systems, 32, 2019.
[20] Gabriel Peyré. Marco Cuturi, and Justin Solomon. Gromov-wasserstenn averaging of kernel and distance matrices. In International conference on machine learning, pages 2664-2672. PMLR, 2016.
[21] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
[22] Yongliang Shen, Kaitao Song. Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems, (6:33154-3818 2023.
[23] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595-46611 2023.
[24] Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. Agentbench: Evaluating llms as agents. In ICLR, 2024.
[25] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consisteney improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
[26] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint 1rXiv:1503.02:: 023311 2015.
[27] Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. Tinybert: Distilling bert for natural language understanding, In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4163-4174, 2020.
[28] Facundo Mémoli. Gromov-wasserstein distances and the metric approach to object matching. Foundations of computational mathematics, 11:417-487, 2011.
[29] Titouan Vayer, Lactitia Chapel, Rémi Flamary, Romain Tavenard. and Nicolas Courty. Fused gromov-waaserstein distance for structured objects. Algorithms, 13(9):212, 2020.
[30] Hongteng Xu, Dixin Luo, Lawrence Carin, and Hongyuan Zha. Learning graphons via struc- tured wraseeewasserstiin barycenters. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 10505-10513, 2021.
[31] Yuanyi Wang, Haifeng Sun, Chengsen Wang, Mengde Zhu, Jingyu Wang, Wei Tang, Qi Qi, Zimi Zhuang, and Jianxin Liao. Interdependency matters: Graph alignment for multivariate time series anomaly detection. arXiv preprint arXiv:2410.08877 2024.
12

---

[32] Yuanyi Wang, Haifeng Sun, Jingyu Wang, Qi Qi, Shaoling Sun, and Jianxin Liao. Gradient flow of energy: A general and efficient approach for entity alignment decoding. arXiv preprint arXiv:2401. 12799 2024.
[33] Yuanyi Wang, Haifeng Sun, Jiabo Wang, Jingyu Wang, Wei Tang, Qi Qi, Shaoling Sun, and Jianxin Liao. Towards semantic consistency: Dirichlet energy driven robust multi-modal entity alignment. In 2024 IEFE 40th International Conference on Data Engineering (ICDE), pages 5559-3572. IEEE, 2024.
[34] Hongtengl Xu. of the AAAI conference on artificial intelligence, volume 34, pages
factorization models for graph clustering. In Proceedings
2020.
[35] HuggingFace. Model outputs documentation.
hnttps:/hugginngacee cotea/cee.o/////eeee/eeeeeee /ocs/
transformers/en// http////aainnnasseeseeeeeeeeeeee
2025.
[36] Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang, Ning Cheng, and Tianyi Zhou. Superfiltering: Weak-to-strong data filtering for fast instruction-tuning In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 14255-14277 2024.
[37] Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang. Ziju Shen, et al. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13:9, 2024.
[38] Zhangchen Xu, Yang Liu, Yueqin Yin, Mingyuan Zhou, and Radha Poovendran, Kodcode: diverse, challenging, and verifiable synthetic dataset for coding, 2025.
[39] An Yang. Baosong Yang. Beichen Zhang. Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan
Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15113 15115, 2024.
[40] Mistral AI.
https://huggingffaee cttps//aaccc.ommmssaaaaallal
Mistral- - Smal-mmll-l---- 1etaal1-2BB--- Instruct 2501 2025.
[41] Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell Hewett, Mojan Javaheripi, Piero Kauffmann, et al. Phi-4 technical report. arXiv preprint arXiv:2412.
2024.
[42] Qiushan Guo, Xinjiang Wang, Yichao Wu, Zhipeng Yu, Ding Liang, Xiaolin Hu, and Ping Luo. Online knowledge distillation via collaborative learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11020-11029, 2020.
[43] Fanqi Wan, Longguang Zhong, Ziyi Yang, Ruijun Chen, and Xiaojun Quan. Fusechat: Knowl- edge fusion of chat models. arXiv preprint arXiv:2408. 07990, 2024.
[44] Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoe V Le, Ed H Chi. Denny Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. In ACL (Findings), 2023.
[45] Vikas Yadav, Steven Bethard, and Mihai Surdeanu. Quick and (not so) dirty: Unsupervised selection of justification sentences for multi-hop question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNIIP) pages 2578-2589, 2019.
[46] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations.
[47] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiy preprint arXiv:2110. 14168, 2021.
13

---

[48] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
[49] Wenhu Chen, Ming Yin, Max Ku, Pan Lu, Yixin Wan, Xueguang Ma, Jianyu Xu, Xinyi Wang, and Tony Xia. Theoremqa: A theorem-driven question answering dataset. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2023.
[50] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07733 077322 2021.
[51] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021.
[52] Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gard- ner. Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2378, 2019.
[53] Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2019.
[54] Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction followoin evaluation for large language models. arXiv preprint 2:Xiv:23/10079 1, 2023.
[55] Sebastian Prillo and Julian Eisenschlos. Softsort: A continuous relaxation for the argsort operator. In International Conference on Machine Learning, pages 7793-7802. PMLR, 2020.
[56] Mathieu Blondel, Olivier Teboul, Quentin Berthet, and Josip Djolonga. Fast differentiable sorting and ranking. In International Conference Machine Learning, pages 950-959. PMLR, 2020.
[57] Olivier Bousquet and André Elisseeff. Stability and generalization. Journal research, 2(Mar):499 526, 2002.
of machine learning
[58] Mher Safaryan, Alexandra Peste, and Dan Alistarh. Knowledge distillation performs partial variance reduction. Advances in Neural Information Processing Systems, 36:75229-75227 752588 2023.
[59] OpenCompass Contributors. Opencompass: A universal evaluation platform for foundation models, https://github..com com/ open cowpass/oppccoopaa.aaa.. 2023.
[60] Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation. In seirth Conference on Neural Information Processing Systems, 2023.
[61] Meyer Scetbon, Gabriel Peyré, and Marco Cuturi. Linear-time gromov wasserstein distances using low rank couplings and costs. In International Conference on Machine Learning, pages 19347-19365. PMLR, 2022.
[62] Gabriel Rioux, Záv Goldfeld, and Kengo Kato. Entropic wasstv-a distances: Stability and algorithms. Journal of Machine Learning Research, 25(363):1 2024.
[63] Linfeng Ye, Shayan Mohajer Hamidi, Renhao Tan, and En-Hui Yang. Bayes conditional distribution estimation for knowledge distillation based on conditional mutual information. arXiv preprint arXiv:2401.08732, 2024.
14

---

[64] Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang, Minillm: Knowledge distillation of large language models. The Twelfth International Conference on Learning Representations.
[65] Songming Zhang, Xue Zhang, Zengkui Sun. Yufeng Chen, and Jinan Xu. Dual-space knowledge distillation for large language models. In Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing, pages 18164-18 18181, 2024.
[66] Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. tions. arXiv preprint arXiv:2212.1000 2022.
Aligning language models with self self-generated instruc-
[67] Wei-Lin Chiang, Zhuohan Li, Ziqing Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See lttps://vvicuaa. lmsys. org (accessed 14 April 2023), 2(3):6, 2023.
[68] Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganch Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Supperriaturallinnttuctoooss Generalization via declarative instructions on 1600+ nlp tasks. arXiv preprint arXiv:2204.07705, 2022.
[69] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
[70] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.0m 01068, 2022.
[71] Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, et al. Crosslingual generalization through multitask finetuning. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1; Long Papers), pages 15991-16111, 2023.
NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: The main claims in the abstract and introduction align with the paper's contributions and scope, providing a clear and accurate overview.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings. is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of work in Appendix
Guidelines:
15

---

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-texi system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
The complete mathematical derivation can be found in Appendix A and
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We have provided implementation details in section 4.1 and Appendix D and included reproducible anonymous code repository URL in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data provided or not.
16

---

If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: section we provide the data sources, and in the beginning of Appendix, we include an anonymous code repository URI. for reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
17

---

Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes)
Justification:
We have provided implementation details in section 4.1 and Appendix D
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Given that calculating statistical significance requires additional resources, we did not perform these calculations. Most of our experimental results show improvements of more than percentage point, indicating a certain level of significance.
Guidelines:
The answer NA
that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: In section 4 we provide information about the computational resources used.
Moreover, in Table 2 show the GPU hours required for model training.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
18

---

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics https://neurt wccppbbiic/thicscuulieiiec
Answer: [Yes]
Justification: The research fully adheres to the NeurlPS Code of Ethics, ensuring integrity, fairness, and transparency throughout the study.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: Our method focuses improving the performance of general models, which does not directly cause social impact. Additionally, we will provide licenses for our code and model weights.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: We will provide licenses for code and model weights in the future.
Guidelines:
19

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [No]
Justification: Based on the double-bling review process, all code we provide now has been anonymized and is intended only for reviewers check and reproduce results.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [No]
Justification: Based on the double-blind review process, all code we provide now has been anonymized and is intended only for reviewers check and reproduce results.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
20

---

Answer:
[NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non stanndaddrrd components.
Please refer to our LLM policy (https://neut) / for what should or should not be described.
21

---

A Appendix Proof of the Approximation
Step I: Gtromo--Wsseeve Distance Definition
Consider two graphs with pairwise distance matrices within the respective graphs
C E R and D Rm
and associated node distributions a E A and b E A", where A" denotes the probability simplex in I.. The squared Stomov-WWsserse (GW) distance is defined
GW(
Cc 1 min yell(a.b) A.A.
Dia 7A 75t.
with the transport plan y E Raxm satisfying the marginal constraints
y = and 1, b.
Because of the inherent fourth-order interactions, computing the GW distance exactly prohibitively expensive. Our goal is derive closed-form approximation by compressing the high--dmensional structural information into node-level features and then aligning these features via a sorting-bass based matching strategy.
Step 2:
Dimensionality Reduction
The key observation is that the full distance matrices C and D encapsulate the network structure, but significant part of this information may be captured by summarizing each node's relationships in single scalar or low-dimensional vector. For example, one may define the node features by node degrees as:
172
(c(1)
Cj and SD(k)
DRa-
m
Alternatively, might use the row norm features derived from diffusion kernel. In any case, the function if serves to reduce the nigh-dimensional distance information into a quantity that reflects the "importance" overall structure of each node. In work, apply the node features defined by node degrees.
Step Approximation of Gromov- Wasserstein Distance
Once the node-level features fc and fD are extracted, we want to approximate the GW formulation as follows:
- (n(k) GW(C, fo(()) - fn(()) na Tyt.
k.j.k.l
It is obtained by approximating the element-wise cost in GW formulation and constraining the gap between the original and approximated formulation by a negligible constant.
Proposition [Approximation of the GW] Let C' Rax" and D E mm be the similarity matrices derived from logit self self-inner products and row-normalized such that Σ, Cij = 1 and Σ, Dri Then the absolute error between the exact and approximated GW distances satisfies:
D)
<
(11)
n2
m
Proof. Let C Raxn, D E Rmxm be two similarity matrices obtained by computing each model's logits' inferinner products and nwrmaaillg so that
CG=11 C3
Du D===1
Dui 2 0.
1=1
Define their row-means and global means:
1 fo(i)
1 Cu.
Cij
PC nu
j=1
22

---

SD(K)
1
DEL
PD
Σ
DEI-
m
1=1
Rdd
Set
A = 11a DDil = (fc(i) - (fol) - fr(()).
We expand each in turn:
Expansion of A.
= Σ
C2 + Di -2 Dm)
liki
m
2
Σ 2 + n
Σ
Dzs -2 E Diy
iiiki
m
Σ
2 + 2
Σ
D 2 2m pC PD
Expansion of B.
B = (f(c())) -fo()))f((())
(fo(()) fr(()))
(fol() - fr(()))
1
(m Es((i [o(()
n Σ {D(K)) (nm (aC---1-0 - pD))) nimm (PC - MD)²
k
Combine A-B. B.
A-B B m² Σ C + n² E 2n -222223 2m² m22m2poHe pCuD n² m2 2DD)
=m? E +n² ] D2 -2n' mn HCHD n²mm² (1C 2UCKD + 1DD)
= =m² 2 Ee +n² ' D2 - ni n2m² 2mm22uz (1ē + 1Â + 2/c/D) + 2n3m² mouu - 2n²m² HCHD
=m² - +
" - n2m² (ve + =b))-
Recall the global Root Mean Square (RMS) variance definitions
1
1 m
(C - mc)3
(Dw PD)
n2
ij
R.
Hence
Σ - n ( +
Σx),
Σ
Da = m² (1T Σz)
ij
kdl
Substitute into A - B:
A - = m 2 (n² ( [/2 + 2 + n² (m (1² + Eb))
n2 122 ( + -13)
= n m2 2 2 + n'm² Nz n²m² n²m + Σz)
Under uniform transport Fix = 1/(m)),
A-B
error
(1Cb Daal _.... 2 )..)))))))))) kjj.k.
Σ
2τ + Em
n²m²²
Since each row of C sums to I,
Ci = 1,
23

---

the worst- per-row variance occurs when one entry is and the rest O:
- - -11-))))-111)) + (n 1) 1/ ) n
2=1
Summing over i 1, ,..., gives Σij (Cij
1
Thus
"
n 1
m
E
n²
m
Combining the above,
- - S E +
11
m
D
n
m2
Because the summation is separable in the pairs (i, k) and (j. 1), define
(
Σ (c( (k) Tik
i,k
Thus, the approximate squared GW distance reduces to
GW²
(A(7))²
Our focus now turns to choosing a transport plan 7 that minimizes this expression. Notably, we only provide the proof of the scalar method (degree property as the feature), since the spectral method can be proved in the same way.
Step 4:
Choosing the Transport Plan via Barting-Based Matching
The transport plan must satisfy the marginal conditions:
Ene = a((),
Σ mi b(k))
For simplicity, assume uniform marginals, i.c., a(i) stochastic matrix with a balanced assignment.
1 and b(k) 1, In this setting, y is doubly
To bypass the computational difficulty of optimizing over all such matrices, we adopt a direct matching approach by sorting the node features. Let
f = sor((fc) and $ =b=t(r((sn))
with the sort performed descending order. natural matching then pairs the i i-th largest entry of Jc with the i-th largest entry of fD-
This sorted matching corresponds to induced transport plan that effectively acts indicator matrix. Under this matching, the transport cost simplifies to
A R
[b(1) - 1b()
where k = (n, m..
Step 5: Final Solving Strategy
By combining the above steps, we obtain the following approximation strategy for solving the GW distance:
D) Σ 100 == min(n,
Here, ft. and 1 denote the sorted node feature vectors (obtained in Step 2, like the row mean or diffusion kernel methods).
24

---

A.1 Discussion
Dimensionality Reduction: By condensing the full distance matrices one-dimensional node features, the original fourth-order optimization problem is transformed into much simpler vector matching problem.
Direct Matching: The sorting-based strategy provides natural one-to-one correspondence between nodes under the assumption uniform marginals, thus eliminating the need for expensive optimization
the transport plan space.
The final formulation is computationally efficient and differen- tiable, making it suitable for token space alignment.
Closed-Form Expression:
Flexibility:: Although we illustrated the process with a simple row-mean (equal the degree property) extractor, alternative feature extractors (e.g., row norm or diffussin- based features) can be used to potentially improve the approximation quality.
B Appendix Error Analysis of the Closed-Form
This section aims to investigate whether the approximate GW distance converges to the true GW distance when the node features become increasingly representative of the underlying graph structure.
B.1 Part Stability of Feature Extraction
We want to show the degree feature extraction (i.c. taking row means of the similarity matrix) in our work Lipschitz continuous, and hence, under mild conditions, the extracted features will converge to the true underlying structural descriptors as the noise decreases or with increasing sample size. We take the row means (degree property) strategy as an example and give the specific analysis, where the spectral strategy can be analyzed in the same way.
B.1.1 Notations
Let C' E Rax= be the similarity matrix, representing graph structure, and define the degree feature of node i by
n
f(i)
Σ
Ci.
n1
This defines the mapping
1 C1,
F R +R, F(C)
where 1 E I is the all-ones vector.
B.1.2 Analysis Lipsehitz Continuity
For two matrices C, C' mx we have
= F(C))
1.)1.
Taking the 2-norm, we obtain
[F(C) - F |(C 1122
Using the inequality (the property of induced norms)
[I(C C') 1|12 ||C -
and noting that ||| = /nn, we conclude:
[F(CC) F(C')l2 < - -
_
1
C CC'l2.
Vn
Thus, F is Lipschitz continuous with Lipschitz constant L
Vn
25

---

B.1.3 Implication for Consistency
The Lipschitz continuity of F implies that small perturbations in C' due to noise in the logit similarities result only small changes in the extracted degree features:
[[((C) F( -
Vn 1C
Therefore, as the noise level decreases or as the number of nodes n increases, thereby refining the approximation, the extracted features F(M) converge to the true underlying structural descriptors of the graph.
B.2 Part 2: Robustness of Sorting
After having proved that the row mean operator is Lipschitz continuous, we need to show that the sorting operator is stable under small perturbations, which means that if two feature vectors close, then their sorted orders are also close.
This has been studied in the context of differentiable sorting soft sort operators 55 56 The core idea behind these studies is that, assuming the extracted feature vector has distinct entries, a small perturbation that is less than half the minimum gap between any sorted entries will leave the ordering unchanged. This means that the sorting operator is locally stable if the input features converge, then the sorted order converges well. As proved in Proposition 2 in [56], "the sorting is differentiable a.e and not only converge to their hard counterparts, but also satisfy some of their properties for all c". Hence, as the estimated features converge, then the sorted order converges as well.
B.3 Part Error Bound for the Approximation
Two primary sources contribute to the overall error in the approximation:
1. Approximation Error. The error incurred by approximating - Duil by ((i, ((,,)) = [f((i) fn (oo()
For all indices (i,j, we have proved that
CCi -
o(i, ((i,k)o(; Tik 7yt
Eapprox
n2
m
2. Sorting Stability Error. Denote by Esort the error incurred by the potential change in the sorted order when the features are perturbed. Under the assumption that the features have positive minimal gap, the sorting operator is stable, or, in practice, one can use a soft sort with known Lipschitz properties, Hence, only small errors are induced in the sorting operator, which is bounded by small Esort (proved in Proposition 2 by Blondel [56] ).
In summary, the error for the approximation of
distance results from
- Csort fapprox
C Appendix Graph Regularisation Bounds
We give proofs for the two Lipschitz lemmas used in Sec. Throughout we assume that the teacher logits are uniformly bounded || do I R, and, without loss of generality, so are the student logits
||S|| < R.
C.1 Uniform stability
Following Bousquet and Elisseeff (57), algorithm A is y-uniformly stable if, for any training set S= (.1...., }. removing one example 2; yields
sup)([A((), (SA S
(12)
Uniform stability directly bounds the expected generalisation gap by 7. Bousquet Elisseeff have also proved properties about 7:
26

---

1. If an algorithm has y-uniform stability, then its generalization error can be controlled in (r)) order of magnitude.
2. is approximately of the order of L/n, where L is a Lipschitz constant of the loss function and n is the number of samples.
C.2 Lipschitz constant of GW loss
Lemma (GW Lipschitz constant), Let (ow ( If 1| & R. then
64\R²
VV&Lowll 1
Low.
D
where is the vocabulary size.
Proof. Recall the squared Saanoo--sssertteinn loss
Low(T, X min PEN((a)))
Σ -Cs(k)) Ps Px Pa. Cs(k,
Cs(k, (Sx -Si)
Let P* denote optimal coupling for the inner minimisation.
Define the distortion tensor
Dr.s(k,1) = Cr(kk Cs(k, I). For coordinate E (1....,D) D}
dLaw
-2 Σ Dr,s(k,1) (Ks(k, k.
Σ Pa PM
asp
OSp
Because |Sa], S R. [D,,s(k, = |(T, - -Ti)² --)=--(3 (Sp ( S 4R².
(A.1)
Then consider the derivative of Cs(k, (Sx - Si)² k, E fl..... D): Taking the partial derivative with respect to the single logit Sp gives three disjoint cases:
1. p k: 8Cs(k,1)/05, =0s, (Sx --)² 2Sx -i))
2. p= (Cs(k, = ds, (S₃ - Si)² 2 (Sx - -S))
3. p p k, Cs(k, does not depend on Sp. hence the derivative is 0.
All three cases are compactly encoded with Kronecker deltas:
(k, 2Sx (Si -dp).
(13)
asp
where Qcp and 8tp denote the Kronecker delta:
1, p,
1, 1=
8kp
8ip
0,
h: + p.
0, 1 1 p.
Assuming every logit is clipped within [- R, Rt,, i.e. |S:| S for i, we have
SSA - Se < |Sal + |S1 < 2R.
Because (ōkp dipl S 1, the absolute value of 13 is bounded by
(k, I)
2 (2R) 4R.
(A.2)
asp
For every (k, l).
P
Σ PT)
m(k) = 1/D2.
(A.3)
ik
Using (A.1)(A.3) and that most one of 8kp, 8ip is non-zero.
(Sxp + 6ip) = 64XR3
&Cow
< 2X (4R²) (4R)
ds,
D2
D
Every coordinate is bounded by 6 proving Lemma
27

---

C.3 Comparison with KL distillation (MiniED)
Lemma 2 (KL loss Lipschitz constant), Let LKL (T, S) AKI(pr ps). pr o ps = o(S) with logits clipped by || R, where the mapping is the sofimax function sending logit vector z to a probability vector on the (D 1) )-simplex A A Then
V s Lx1.ll2 S see D LKL.
(14)
Proof. By definition (L(prllps) = (i) prg))(() - lo (i)] Taking the derivative w.r.t. (s( yields
0KL PT(i)
1
SKL Ups(1)
Ops(i) Ps(j)
mine (i)
Because each logit is in I-R, Rl,
e:
R
e
e22
(i(
2
2
k Ex
DeR
D
Hence [OKL/Ops(i] S 22 D. The soft-max Jacobian J satisfies || 2 5
| = || V ps KL -
28 D S dD
2
where the last inequality uses (so 22 /2 S R for R こ 1; if R small this only tightens the bound). Multiplying by À proves [1)
Comparison. Putting the lemmas together, Lemma 2 alonngiid Lemma scaling of Low with dimension D:
we observe the inverse
(D))
R Oe*D)= LKL
LGw
C.4 Comparison with Wasserstein distillation (ULD)
Lemma 3 (Lipschitz constant of the 1-Wasserstein loss). Let pr = (T) and ps = o(S) be the soft-max probabilities of teacher and student, and define
Lw W (PT.Ps))
where the ground metric satisfies ((i,)))(0,A4 Then
AAVD Lw.
VsLW ll2 - 2
Proof. Using Knnbonovich-Rubiistenn dual:
W (pT.Ps)) = sup
Σ pi (pT(() - (())
1|o/ s SA
For the optimal potential 8², Ups W1 = - hence
V pg W1 22 S vD 12 & AVD.
The Jacobian Jo = app//Os = (6 - -p)) satisfies 1|11221 S because each row has -nom
< 2.
s = Vn Willa < | VV Willa < AVD.
2
Then multiply by the scale A.
AA AA
VsLw |12 = >l|VsWall, <
2
28

---

C,5
Comparison
Combining Lemma
T Lemma 2
and Lemma
we obtain the following relationship between
Lipschitz constants:
64XR
AAVD
Ae DD D,
D
2
for sufficiently large vocabulary size and moderate logit range R.
This comparison shows that replacing the KL. loss with the Coomov-Wassersteiee loss results in the smallest Lipschitz constant, which implies the most stable learning process. As the Lipschitz constant related to the uniform stability bound ? < L/n, a smaller Lipschitz constant translates directly to a tighter generalization gap, improving the model's ability to generalize on unseen data. Thus we can obtain the following proposition:
Proposition 2 (Lipschitz constants comparison) We have the following relationship between the Lipschitz constants (L) for the Gromov- Maanoo-Wasserstein, Wasserstein, and KL losses:
R D)
Low 0
<
=w=D(VD)
LKL= O(E
for sufficiently large vocabulary size D and logit range R, which means the GW-guided training have tighter generalization bounds.
Thus, replacing GW results in the smallest Lipschitz constant, leading to better stability and tighter generalization bounds.
C.6 Birs-variance decomposition
Following [58).
Egen
Bias (T,S)
()
+ Var
teacher teacher-guided
St//
Teacher logits fix the bias term; graph regularisation lowers the Lipschitz constant (Low < LKI.) and thus the variance term, yielding tighter generalisation bound, especially in low-data or cross domain settings.
D Appendix Extended Experimental Setup
D.1 Fusion Datasets details.
We construct multi-task dataset comprising 130k examples across general reasoning, mathematics, and coding. The general data are sourced from Infinity-Instru [36]
Mathematics. The mathematical dataset we use comprises 39k pairs of math questions and cor- responding answers. The questions are sourced from the Numina.M ath 1.5 dataset, while the answers are distilled from the DeepSeek-RR R1-67 -67 IB model by the AM tean FurinaM ath_ 1_5 represents the second iteration of the widely acclaimed NuminaMath 37 dataset. It offers high- quality data for competition-level mathematical problems across various domains, including Algebra, Geometry, Combinatorics, Calculus, Inequalities, Logic and Puzzles, and Number Theory.
dataset [38], which contains approximately 268k samples. Each sample was fed into our pivot model, which was prompted generate 5 random responses per input. The generated outputs were then passed through a sandbox-bars evaluation system. For each sample, at least one of the five responses failed, the sample was flagged for further consideration. From these flagged samples, filtered and selected 39k nigh-quality examples for further distillation.
Coding. In the code generation domain, we utilized the KodCode-VI-SFF
RI-Distilled-1..
29

---

Training Details. We train with C-AdamW and cosine decay for epochs, using early stopping on the 4th. Fusion is performed with extlne-extracted hidden states (before the 1m_ head) reduce GPU cost, requiring 1.5TB of storage. Fusion uses batch size of 16 on 8x80GB NVIDIA H800 GPUs for 20 hours. We adopt early stopping at epoch (of 5), learning rate 10-6, and default ULD weight À 0.5, GLD weight À 0.001.
Offiine Teacher Loading. For InfiGFusion, we extract hidden states (before the am_head) from source models and store them ( 1.5TB per run). Only final layers are retained online, reducing memory and speeding up training.
E
Appendix EEvaluation Prompt
To enhance the robustness of answer extraction under the regex-based evaluation framework of OpenCompass [59] and EvalPlus [60]. we systematically refine the prompts used in several benchmark datasets. These tailored prompt formats are designed facilitate precise output matching, mitigating ambiguities that often arise from model generations. The revised prompt templates corresponding to each dataset are presented in Table 7 which details how task instructions and answer formats are standardized to align with OpenCompass automatic evaluation pipeline.
For datasets such TheoremQA and HumanEval, retain the original prompt configurations, adhering to their respective community-adopted evaluation protocols. This ensures consistency with prior works and preserves the validity of established benchmarks. For MBPP, we utilize EvalPlus (60) for more rigorous assessment of ALM--gnnrated code, providing enhanced reliability in functional correctness evaluation.
Table 7: Prompt format used for different datasets.
Dataset
Prompt Format
IFEval
directly give the correct answer: Question: {question\\nA (textA)\nB (textB) AnC (nextC)\nD (textD) AnDirectly give me the correct answer option, and then explain: {etx} AnQuestion Which ending makes most sense? lonsellnDirectly give me the correct choice, you can further explain it or not. AnA (A (B)AnC (C)nn (nYou may choose from 'A', 'B', '0', 'D', AnAnsver
[prompt\AnPlease
ARC-C
Hellaswag
BBH
Follow the given examples and answer the question. An (_hint]\nq ((input)) Let's think step by step. You will asked examples of passages and Q\&A are provided below. An (drop_examples)/rr in ## Your Taskln --An {promptMnnhhnn step by step, then write line of the form "Answer: VANSWER" at the end of your response. (_hint)\nQusstiin
DROP
a passage
answer question. Some
MMLU
((input))\nd ((A)) ((B)) (nD ((D)) AnFor simple problems: \nDirectly provide the answer with minimal explanation. An \nFor complex problems: \nUse this step-by-step format: An #### Step #**# Step 2: [Concise description]W. An [Brief explanation]]n In
[Concise description] [Brief explanation] \n
inñegardless of the approach, always conclude with: AnThe ansver is \nwhere the [the'\answer [etter) is one of A, B, C or An \nLet 's think step step.
GSM8K
equestion/\nPleaat reason step by step, and put your final answer within (\boxed((1).
MATH
Ceroblem/\nPleace reason step by step, and put your final ansver within \boxeed(())
30

---

Table 8: Performance on BBH reasoning tasks. Tasks marked in bold involve complex relational rea- soning, multi-hop inference, or semantic alignment, where InfiGFusion shows notable improvements over token-level fusion baselines.
BBH Dataset
Qwen2.5 Mistral-24B Phi4-14B DeepSeek Phi4-14B (SFT) InfiGFusion
bbh-temporal_ sequences
sequeeeesssqquueeeess
98.8
98.8
99.6
66.8
98.8
100
abt-disanbiiguaton..y
78.8
75.6
82.4
74.8
79.6
76
bbh-date, understanding
90.4
86.8
94.8
50.4
85.2
bbb-tracking shwpgggsssseeled_eeeeee shuflled_oojeetsss objects_ wwtetthrreeeeeeeeeeeeeeeeeee
99.2
99.6
97.2
hbh-penguins_iie in table
90.41
97.95
98.63
69.86
93.84
97.26
abt-geometric__hapee
58.8
49.2
56.4
71.6
56.4
62.4
bhh-snarks
84.27
85.96
61.8
47.19
70.22
79.21
boh-ruin_name names
82.8
76.8
88.8
39.6
88
88.4
bbb bbb-tracking shuffied objects seven, objects
80.4
96.4
94,4
80.8
90
96.8
bbb-tracking_shulle laangn__shllle___bge wbwectss_ive__bppppebeepp five__ijjects
79.2
99.2
96.8
82.8
95.6
96.8
abb-logical_dddine deduction, throc_trrrooo throe_objects objects
97.6
98.8
98.4
83.2
96.4
97.6
bbb-hyperbuton
97.2
98.8
49.2
53.2
99.6
obb-logical_deductinn. deduction fiie__bjectt
80.4
85.6
86
92.4
ith-logiia_deeductinn
deduccioon seven_ohjects
68.8
62.8
88.4
83.2
88.8
89.2
Nob-movie_rccommenddaee recommendation
72.8
78
58.8
60.8
62.8
43.6
beasalient_translatiin Beaee__rrr___eee dere_deetttion
70.4
63.2
65.2
54.4
64
67.2
bbh-rcasoning about_ colored, objects
93.6
90
96.4
87.6
96.8
96.4
bbh-multistep,
sitphtitttppariimmmttti._.c wwwaarrrttteeeeee two
96.4
93.2
64
81.6
62
99.6
bbh-navigate
97.2
80.8
96
51.6
82
96
bbh-dyck_langura languaggss
35.6
11.2
13.2
abh-word_sorting
32
36.4
0.4
9.6
59.2
hhh-sports, understanndnng
79.2
91.2
72.4
89.2
92.4
hbh-boolean_ewwe eexpressioosessss
55.6
87.2
28.8
8
66.4
98.4
bob-object_counttin
counting
89.6
95.2
98.8
99.6
99.6
98
54.8
73.2
0.4
0.8
0.8
91.6
bbh-causal judgement Lunggmmml
43.85
68.98
32.99
45.35
35.83
70.05
bbh-web_of Jies
99.2
100
100
93.6
100
100
F
Appendix Comprehensive Reasoning Performance
F1
Analysis on BBH: Semantic Dependency in Complex Reasoning
We conduct comprehensive evaluation of InfiGFusion on the BBH benchmark. which consists of diverse and challenging reasoning tasks requiring multi-step logic, relational alignment, and semantic consistency. These tasks pose significant challenges for token-level fusion methods due to their reliance on token co-activation patterns and internal reasoning structures.
Remark. The use of "internal reasoning" refers the relational structure among semantic dimensions in the logit space. We align inter-logit relation graphs via Gromov- Wasserstein, providing a structural inductive bias for reasoning- heavy tasks, without explicitly modeling the reasoning steps.
Semantic Dependency Fusion Matters. InfiGFusion demonstrates clear advantages tasks where reasoning relies on inter-token dependencies rather than isolated token probabilities. For example, in Logical Deduction (5 objects 7 objects), InfiGFusion outperforms the SFT baseline by + +1.6% and +0.4%, respectively, indicating better preservation of relational structures. In Tracking Shuffled Objects tasks (five and seven objects), InfiGFusion matches exceeds the best-perforning models, showing that graph-based alignment can maintain structural consistency in object tracking.
Superior Multi-hop and Arithmetic Reasoning. Tasks such Multistep Arithmetic Two (+36.8% over SFT) and Causal Judgement (431 155%) highlight InfiGFusion's strength in capturing multi-hop reasoning and causal dependencies, which notoriously difficult for token-level methods. These improvements suggest that GLD loss effectively aligns higher-order reasoning structures across source models.
Recovering from Source Model Weaknesses. InfiGFusion mitigates the weaknesses of individual source models in cases where token-level fusson underperforms. For instance, in Boolean Expres- sions, InfiGFusion achieves 98.4%, significantly outperforming phi4 SFT (66.4%) and other basclines,
31

---

by aligning semantic relationships between logical operators through graph structures, Similarly, in Dyck Languages a task requiring bracket matching and structural hierarchy InfiGFusion closes the gap (+23.6% over SFT), showcasing its advantage in structure-sensitive tasks.
Robustness in Semantic Alignment. On tasks like Ruin Names and Reasoning about Colored Objects. InfiGFusion maintains top-tier performance, matching or slightly exceeding SFT results. This indicates that InfiGFusion not only aggregates external knowledge but also preserves the pivot model's strengths, avoiding catastrophic forgetting.
Challenges and Observations. While InfiGFusion excels in structure-dependent reasoning, shows less improvement in tasks dominated by shallow lexical patterns (e.g.. Word Sorting, Sports Understanding), where token-level matching suffices. This suggests that InfiGFusion's strength lies in modeling inter-token dependencies rather than surface-level token alignment.
Overall, InfiGFusion outperforms token-level fusion methods on BBH tasks requiring structural reasoning, multi-step inference, and semantic consistency. By explicitly modeling logit-space semantic dependencies, InfiGFusion enables robust and interpretable fusion of heterogeneous LLMs.
F.2 Analysis on MMLU: Fusion of Knowledge and Reasoning Behaviors
We further evaluate InfiGFusion on the MMLU benchmark, covering 57 sub-domains across STEM, social sciences, law, and humanities. Unlike BBH, which emphasizes complex reasoning chains, MMLU focuses on factual knowledge, domain-specific reasoning, and multi-field coverage, offering complementary perspective to assess fusion effectiveness.
Retaining Pivot Knowledge, Enhancing Structural Alignment. InfiGFusion demonstrates strong consistency with the pivot model's strengths, while further integrating structural reasoning from source models. On factual knowledge tasks like High School Computer Science (95%) ) and US History (91.67%), InfiGFusion matches slightly improves over Phi4 SFT. This indicates that structure-awee aware fusion preserves core pivot capabilities without degradation.
Advantages in Structure-sensitive Tasks. InfiGFusion outperforms Phi4 SFT on structure- dependent categories such as Abstract Algebra (+2%) and Security Studies (+1.63%), confirming its effectiveness in aligning semantic dependencies. Similarly, in tasks like Machine Learning and Computer Security, InfiGFusion maintains competitive results, highlighting its ability to integrate relational knowledge from multiple sources.
Balanced Performance in STEM-heavy Tasks, While DeepSeek-RR exhibits domain-speciffe advantages (c.g.. College Mathematics: 93%), InfiGFusion remains competitive (81%). achieving balance between generalization and specialized knowledge. On Clinical Knowledge and Medical Geneties, InfiGFusion slightly trails SFT but still improves upon the raw Phi4 model, reflecting its capacity to enhance reasoning fidelity without overfitting to narrow domains.
Stable Gains in Social Sciences and Law. In domains like International Law, Global Facts, and World Religions, InfiGFusion maintains accuracy on par with SFT, demonstrating robustness across disciplines. Notably, Sociology, InfiGFusion surpasses all baselines (90.05%), suggesting effective fusion of nuanced reasoning patterns.
InfiGFusion's 's gains are most prominent in tasks requiring structural reasoning and multi-source knowledge integration, while its performance on pure factual recall tasks remains on par with SFT. This validates our design choice of graph-based semantic alignment for model fusion, enabling balanced and interpretable improvements.
G Appendix Relative Error of GW
Setup. To better analyze the relative error the proposed GW, we quantify the scale of the GW distance and its relative error representative BBH subsets using 14B student and 130K training examples. We compare three objectives: (i) our sorting-based closed-form approximation (O(n log )), (ii Sinkhorn GW (O(n² log n)). and (ii)) Entropic GW (O(n² n)). Following [61
32

---

Table 9: Performance on MMLU sub-tasks. InfiGFusion excels in structure-sensitive reasoning while maintaining robust general knowledge performance. Bold categories indicate tasks where structural dependencies and multi-source knowledge integration critical.
MMLU Dataset
Qwen2.5 Mistral-24B Phi4-14I DeepSeek Phi4-144 14B (SFT) Infiinusiion
lukaemon, mumlu aunlu_college. ceolleelleeooeeee biology
90.97
96.53
94.44
92.36
lukaemon mmlu, mmmlu_college__mm www___leeeeeeeiissseeee
72
66
63
lukaemon_ mmlu, mmllucoollege computer___scceeeeeeeccccceeeeee _sseence
76
86
82
85
lukaemon_ mmnuummuaee ceanllueollegeeeeee cwwllgeemmatemmaiii mathematics
80
72
79
93
77
81
lukaemon_ Makamoo__mmuu....m _mmlu_college___ _ew_llel___phhee _pcceesss_cssssseee
78.43
89,22
94,12
92.16
88. 24
84,31
lukaemon_ mmluu wtt__eeeetrcal_eeniiieee engineering
77.93
78.62
82.07
80.69
82,07
82.76
Iukaemon mmlu autuosssooooy
87.5
94.08
90.79
92.76
89.47
90.79
lukaemon_ mmlu, htt_m_mmmmmlttoooe anatomy
76.3
79.26
82,96
74.81
80.74
78.52
lukaemon__ hat___muu____aa abstract_ulgeete
73
65
82
86
88
lukaemon, minlu, machine learming
67.86
70.54
80.36
77.68
81.25
78.57
lukacmon_ immlumeeie clinical knowledge
84.15
87.17
84.91
83.77
89.43
87.55
Iukaemon_ mmlu global facts lukaemon, munlu management lukaemon,
86.41
85.44
86.41
84.47
87.38
85.95
87.25
87.91
84.31
lukacmon mmlu marketing
90.17
92.74
89.32
92.74
95.3
Iukaemon mmlu professional accounting
75.89
73.4
86.17
78.72
84.75
81.91
lukaemon_ munuu high baggh_schooo_gggg geography
87.88
87.88
90.4
88.89
91.92
89.9
lukacmon_ mmlu intuutnttaaiinal...... Jaw
81.82
81.82
91.74
82.64
90.91
90.91
Iukaemon mmlu _aoa__oraaa__ scenarios
71.96
68.6
75.75
73.41
74.75
73.97
lukaemon_ mmlu, httpl_oompuu__eeeeeee
81
85
85
83
87
lukaemon_ m_mluu_______ high_schoo microeconomics niihoooccccos
88.24
90.34
95.38
94.96
96.22
96.22
lukaemon_ mmluu www.epssooooll.eeee.. law
56.65
61.73
67.28
61.67
64.99
65.71
Inkaemon, mmlu medical genetics
89
93
92
lukaemon_ mmluu profeesioonall.... patesssoooalppyhllgogoooe psychology
80.07
82.19
85.78
86.6
84.97
Iukaemon_ mmlu_ wtt_uummummmuuuudioaeee
jurisprudence
85.19
92.59
86.11
lukaemon, mumlu, world religions
89.47
lukacmon mmlu philosophy
79.74
80.06
84.89
83.92
82.96
Iukaemon_ mmlu viroloyy
52.41
49.4
53.01
54.22
53.61
54.22
lukaemon munlu high./hhhhh_hheeeeeeoooooo cheemissy
76.85
83.25
90.15
88.67
88.67
84.24
lukaemon_ munu public relations
70.91
69.09
77.27
74.55
79.09
75.45
lukacmon mmlu hasp__ihh__hhhheeeeee hechh_shooo__aceecee hcl.ooooooccccccoooiiis macrocconomiss
84.87
84.1
89.74
88.72
90.51
91.03
Iukaemon mmlu haman_asseeeatee sexuality
83.21
88.55
86.26
86.26
87.79
85.5
lukaemon, mmlu, alementary_matheeeee mathematics
95.24
94.97
97.09
97.35
96.83
95.24
lukaemon_ mmlu high_school____.. Bacp__scho___ppee phyysics
78.81
76.82
85.43
84.11
87,42
84. 11
Inkaemon mmlu high_schoo seoh_sscoo___ouuterr computer, science
92
92
96
93
95
lukaemon mmlu high_school european history
81.82
84.24
84.85
83.03
83.64
lukaemon_ mmlu business_com ethics
82
lukacmon_ mmlu moral disputes
76.88
81.79
83.24
81.5
lukaemon, mmlu high school, statisties
88.89
lukaemon mmlu miscellaneous
91.57
94.25
93.87
93.74
91.7
lukaemon_ mmlu fomlu_formal____m formal_loge logic
68.25
66.67
77.78
91.27
78.57
74.6
lukaemon_ mulu, highhhhhhhhhhaeeeeeeeeeee school_ govermment and politios
93.78
95.34
95.85
96.37
96.89
96.37
lukacmon mmlu muml__prchistory prehistory
86.11
83.64
85.8
83.64
87.35
87.65
Iukaemon mmlu, security_stuiee studies
77.96
78.78
77.14
78.37
79.59
81.22
lukaemon, munlu high school_hhbieeeee biology
90
90.97
94.19
92.58
92.9
94.19
lukaemon, mmlu logical fallacies
85.28
84.66
86.5
85.28
87.73
87.73
lukacmon_ umlu high segoo_schoo__ooooe
history
87.76
88.61
89.87
89.45
88.61
89.87
lukaemon mmlu professional medicine
84.19
92.28
91.18
84.19
92.28
91.54
lukaemon, munuu highhsshoon school, mathematics
85.93
84.07
92.22
97.04
90.37
90
lukacmon_ mmlu college medicine
79.77
83.82
84.97
88.44
85,55
85.55
lukaemon, mmlu high school, history
91.67
lukaemon, mmlu. wwwlumummiiillllaaaa sociology
86.57
85.57
89.05
84.58
88.06
90.05
lukaemon wwwmmmmmmaeameeeeaeaa cconometrics
60.53
71.93
65.79
71.93
67.54
lukaemon, mmlu hiamllhhghhh__ high_school school_oooseee
heh_scooossycchllggy psychology
90.64
92.84
94.31
94.86
95.6
Iuknemon mmlu human _aiin
73.99
75.34
80.72
78.48
78.48
78.92
lukaemon mmlu so__foreign_pol pplicy
89
88
92
93
91
lukaemon, mmlu camceptunl_physiie
88.09
86.81
89.79
91.06
90.64
88.09
33

---

Method Logical Deduction Multistep Arithmetic Causal Judgement Dyck Languages
Our Appreximation (Sorting Based, loglo)))
GW Distance
0.0356
0.0614
0.0445
0.0210
Abs. Error Bound
0.00015
0.00015
0,00015
0.00015
Relative Ervor (%)
0.421
0.244
0.337
0.714
Perforrmnce (%)
89.2
99.6
70.0
94.4
Sinkhom GW (O(nnO logn)) m))
GW Distance
0.02145
0.0618
0.0449
0.0213
Abs. Ertor Bound
0.00009
0.00009
0,00009
0.00009
Relative Frvor (%)
0.419
0.145
0,200
0.422
Performance (8)
89.9
99.2
70.4
94.1
Entropic GW (O(n"logn)
GW Distance
0.0314
0.0617
0.0448
0.0212
Abs. Error Bound
0.00012
0.00012
0.00012
0.00012
Relative Error (%)
0,382
0.194
0.267
0,566
Performance (%)
89.7
99.6
70.7
93.9
Table 10: Scale of GW distance and relative error on BBH subsets (14B model, ~330 data).
Serwed
Sreathed cas Caldetters
S00e
Figure 5: Comparison of WD and GW distributions during fusion. Left: before distillation; Middle: after token-level WD optimization. WD reduces significantly, but GWD remains largely unchanged, indicating semantic dependency misalignment; Right: after GLD optimization.
62 we report a certified absolute error bound and compute relative error as RE Abs. Errer Bound X
GW Distance
100%. We also list the end-task performance (%) obtained when optimizing each objective.
Findings. (1) Distance scale and RE. GW distances BBH fall between 0.02 and 0.06. With the certified absolute bounds, relative errors are consistently below 1% (Table I0). supporting the suitability of using the approximation
training objective. (2) Approximation quality vs. cost. Sinkhorm/Entropic GW yield slightly tighter RE than the sorting- based objective, but with O(n" log n) cost. Our sorting- based objective offers near-parity performance while remaining O(n log n) and thus scalable. (3) Empirical consistency. The sorting- absolute error aligns with the expected (1/nn) rate at operating regimes (e.g., = 150K, z 100K), while competitive end-task performance validates training usefulness.
On larger models and time gaps. The section 4.7 Table TO and expanded distillation experiments with Table [2 give further discussion. While on smaller students the wall-clock gap between Sinkhorn and our sorting- based objective can appear modest, the asymptotic gap widens with scale. Concretely, the effective problem size n grows with sequence length, batch size, and active logit support (e.g., top-k neighbors), making Sinkhorn/Entropic O(n" log costs rise superlinearly, whereas our objective scales as Onn log n). Empirically, on larger models with more dimensionality, observe proportionally larger savings in GW-side time while maintaining comparable downstream performance.
At the scales of interest for LLM fusion, GW distances are small but stable; certified relative errors under our approximation remain < 1%, and the O(n log n) objective delivers the right trade-off between fidelity and throughput, with advantages that grow with model/sequence scale.
34

---

Condition-dependen Logic (Object Tracking)
Task Type: Sequential state updates under conditions
InfiGFusion Behavior
Baseline Behavior
"Follow the given examples and answer the question. Andnsver extremely simple questions about the colors of objects on a surface. \n\nQ: On the nightstand, there is a red pencil, a purple mug, burgundy keychain, fuchsia teddy bear, black plate, and a blue stress ball. you see green bracelet, fidget spinner, red dog leash, and orange pen. Hou many objects are neither black nor blue?\n0ptions: (A) zero\n(B) oneln(C) two)n(D) (E) (F) fGve'n(G) sixinA: Let's think step by step. An"
"prompt";
\n\nQ: On the floor, purple cat toy, a brown pair of sunglasses, black
"origin.predictionn: "According question, the objects on the floor are as follows: a green bracelet, a purple cat toy, a brown pair of sunglasses, black fidget spimner, a red dog leash, and an orange pen. pen.\n\uWe \n'nWe need to count how many objects are neither black nor blue. AnAn1. Green bracelet: Not black blue count\n2. Purple cat toy: Not black or blue 1n3. Brown
this
"Let's analyze the question step by step: lnlp:\ objects on the floor are:\nt. green bracelet(n2. A purple cat toyln3. A brown pair of sunglasses\n4. A black fidget spinner\n5.
prrigin,predcction
A red dog leash\nn.. An orange pen ln'we' need to determine how many objects are neither black nor neither blue. An\n- The green bracelet black nor blue. An- The purple cat toy is neither black nor blue. \n- The brown pair of sunglasses is neither black nor blue. An- The black fidget spinner is black, it is excluded. An- The red dog leash is neither black nor blue. An- The orange pen is neither black nor blue. \n'nCounting the objects that are neither black nor blue, we have: \n1. Green braceletin2. Purple cat toy Brown pair of sunglasses\n4. Red dog leash Orange pen'nnnnThers are 5 objects that are neither black nor blue. \n/nTherefore, the answer is (F) five.", "correct"; true
pair of sunglasses: Not black or blue countind, Black fidget spinner: Black
+ do not count Red dog leash: Not black or blue count Orange pen: Not black blue count the objecta that neither black nor blue are: green bracelet, purple cat toy, brown pair of sunglasses, red dog leash, and orange pen. That makes 5 objects.\alnThe \n/nThe answer is (F). ", "correct"; true
Table 11: Case study on condition-dependent object tracking logic.
H
Appendix -Distribution Analysis of WD and GW During Fusion
To better understand how InfiGFusion improves semantic alignment, we visualize the evolution of Wasserstein Distance (WD) and stages of fusion (Fig. 5).
Distance (GWD) distributions across three key
Experimental Setup. We compare: (1) The initial distribution before distillation (Left), (2) After token-level WD optimization using ULD loss (Middle), (3) After full GLD+ULD optimization (Right). Each histogram reflects pairwise distances between pivot and source logits across sampled instances.
Why This Matters: WD measures the marginal alignment of token logits, while GWD captures relational (dependency-leve)) differences between token dimensions. Existing fusion methods (e.g.. ULD) optimize WD but ignore semantic dependencies, making GWD a critical metric for evaluating structural alignment.
Before distillation (Left): Both WD and GWD exhibit large values, indicating poor alignment at both token-level and structure-level. After ULD optimization (Middle): WD reduces significantly, showing that token-wise distributions are aligned. However, GWD remains largely unchanged, revealing persistent mismatch in semantic dependency patterns. This validates our hypothesis: teken-level objectives fail to capture cross-token interactions essential for reasoning consistency. After GLD+ULD optimization (Right): Both WD and GWD distributions shift leftward. Notably,
35

---

GWD exhibits clear reduction, demonstrating that GLD successfully aligns relational structures in logit space, complementing token-level alignment.
I Appendix Case study
We provide additional results compared between InfiGFusion and Phi-4 SFT in Table 11
J Appendix Additional Discussion
Our approach focuses on relational consistency, aligning the semantic structure of logits across sources, without an explicit mechanism for resolving, faciual conflicts between models. In practice, when source models encode contradictory knowledge, fusion can produce outputs that are structurally coherent yet factually incorrect. While the framework naturally accommodates extensions such as confidence estimation and external verifiers, our preliminary trials with entropy-based source selection [ and Bayesian conditional alignment 63 yielded only marginal gains our scales and benchmarks.
We view conflict-aware source re-weighring and selection complementary to GLD rather than a substitute. Promising directions include per-claim uncertainty calibration, verifier-in thop -oop pipelines (e.g., retrieval factuality checks), and adaptive top-k pruning conditioned on detected conflicts. A principled, high-yield conflict-resolution module is thus an important avenue for future work, whereas this paper concentrates on semantic structure alignment and stronger GLD objective.
K
Appendix Additional Distillation Experiments
To verify the generality of our Graph-on-Logits Distillation (GLD) loss beyond model fusion, we conduct additional experiments in the traditional knowledge distillation setting, where a compact student model learns from a single larger teacher model. These experiments assess whether GL.D can also improve response quality under extreme model compression (from 7B to sub- 1B scale).
The distillation performance of GLD is evaluated via instruction-followinng accuracy averaged over five random seeds. For distillation, follow prior work and use Dolly-15k [64] for training. We follow [65 that evaluate on four benchmarks: SelfInst (66). VicunaEval [67
Distillation Datasets:
Super Natural Instructions (S-NI)
and the Dolly (64)
Models: For distillation, we distill from LLaMA3-8B. Mistral 7B, and Qwen2.5-7B into student models, including GPT2-120M 669) OPT-350M (70), and Bloomz-560M [7I).
Training settings: Distillation uses LoRA finetuning and temperature 7 1.5. All models use a learning rate of 1 x 10-6 and À 0.5 unless otherwise stated. For all experiments, evaluation metries are averaged across random seeds.
Table 12 presents a comprehensive comparison of GLD against prior distillation baselines across wide range of teacher-student pairs. Across all 27 configurations teachers 3 students methods), GLD achieves the highest average accuracy in nearly all cases, consistently outperforming both MinED and ULD.
Notably, even when distilling into extremely lightweight students such as GPT2-120NG GLD shows robust generalization challenging reasoning datasets like SelfInst and VicunaEyal. For instance, when distilled from Mistral to GPT2, GLD yields an average gain of + +1.39 over ULD and +1. +1.39 over MinED, despite requiring fewer GPU hours than ULD. Similarly, from LLaMA3 to OPT-350M, GLD improves +0.53 over the best baseline with competitive efficiency.
This performance shows that GLD's structure-aware objective can preserves inter-token relationships critical for multi-step and relational inference, particularly impactful for small-capacity models where token-level alignment alone often fails. Compared to ULD, which aligns logits dimension-win wise using approximated Wasserstein distance, GLD additionally distills dime-siimnsionall co-activations via logit graphs, serving as a richer supervisory signal. Its effectiveness across diverse teachers (Qwen, LLaMA3, Mistral) and architectures (GPT-style, OPT, Bloomz) highlights GLD's compatibility with heterogeneous generation styles and token spaces.
36

---

Table 12: Performance of different distillation methods across teacher-student pairs. Rouge-L is averaged over 5 different seeds on 4 instruction-followe benchmarks. Bold indicates the best performance, Improv. denotes the relative performance gain of GLD over the other baseline.
Teacher
Model
Methods
Times/h
Dolly
Selfinst
VicunaEval
S-NI
Average
Improv.
Qwen 2.5-7B
28.7110.31
26.01:0.43
21.32:0.48
42. 42.5622.32
29.65s0.39
Teacher
L.aama3
SFT
32.05a0.38
25.04a0.71
20.68s0.47
40.73a0.29
29.6340.46
Mistral
31.4660.34
25.1540.73
20.5740.29
37.56+0.40
28. 28.6990.44
MinED
6.5
20.5520.15
10.8840.36
14.9010.63
22.12-0.36
17.11=0.38
3.48%
GPT2-120M
ULD
2.5
20.8810.56
I 11.0240.33
15.45:0.36
21.2420.37
17.15:0.41
3.27%
GLD
3.8
21.3810.38
11.34x0.42
15.73+0.15
22.38:0.30
17.58:0.31
MinED
5.0
22.6240.34
12.3740.45
15.52+0.30
21.99a0.12
18.13+0.30
0.73%
Qwen 2.5
OPT:350m
ULD
4.5
22.36+0.29
11.38+0.50
14.61+0.36
21.80=0.59
17.54=0.44
4.1 155
GLD
22.80+0.48
12.61
15.77=0.67
21.85=0.33
18.26:0.51
MinED
22.86:0.56
11.78+0.34
14.82+0.39
22.83=0.30
18.07=0.40
1.37%
Bloomz-560m
ULD
22.5120.32
12.4820.44
14.0820.45
82=0.21
17.97=0.36
1.93%
GLD
22.8320.16
12.5220.14
14.7420.31
23. 23.1930.13
18.32=0.19
MinED
2.9
19.93+0.36
10.1540.24
14.3820.43
19.78=0.23
16,0620.3. 32
3.44%
GPT2-120M
ULD
2.5
19.6620.49
9.8420.31
14.59+0.50
19.77.0. 14
15.97 36
4.06%
GLD
2.8
20.97=0.28
10.81=0.55
14.9710.37
20.09:0.24
16.6920.36
MinED
5.3
21.25
10.0640.38
15.28+0.41
16.41 19.04=0.32
2.18%
LLama3
OPT-350m
ULD
4.4
21.08a0.56
9.85+00.42
14.89+0.50
19.13+0.58
16.24=0.52
3.25%
GLD
4.7
22.01=0.31
10.97=0.33
14.87+0.46
19.21=0.25
16.77=0.34
MinED
4.7
20.16+0.21
10.81
13.36+0.44
20.62+0.11 17
16.24=0.25
2.97%
Bloomz-560m
ULD
5.3
20.69+0.49 49
10,50=0.64 64
12.97+0.61
20.71=0.18 18
16.22=0.44
3.10%
GLD
5.5
21.07=0.47
10.82+0.17
13.41=0.29
21.58:0.24
16.72:0.29
MinEi
20.47+0.62
10.5240.79
15.36+0.27
20.59+0.54
16.74a0.56
8.34%
GPT2-120M
20.00+0.52
10.53+0.81
15.29+0.20
10.57+0.16
16.60=0.4p
9.23%
GLD
21.90=0.31
11.89:0.41
15.99#0.31
22.74=0.27
18.13+0.33
MinED
22.67+0.29
11.9620.7 74
15.85=0.65
22.55=0.20
18.26=0.44
3.63%
Mistral
OPT-350m
ULD
22.45+0.38
11.1920.88
15.41=0.60
23.1720.32
18.06=0.55
4.79%
GLD
23.7220.36
12.73:0.30
15.1420.53
24.11=0.28
18.92:0.36
MinED
3,9
22.73+0.18
12.30+0.37
15.58:0.16
13.97=0.13
18.65+0.21
2.49%
Bloomz-560m
ULD
22.2110.40
12.2340.55
15.1
23.6420.37
30:003333
4.44%
GLD
23.2210.38
12.87.0.49
15.9240.29
24.3300.11
19.11=0.32
37


Reviews:
[{"review_id": "2zj5qsqs1p", "reviewer": "Reviewer_BzTF", "summary": "This paper introduces INFOBLEND, a caching system for accelerating inference in Multimodal Large Language Models (MLLMs) by enabling position-independent reuse of key-value (KV) caches. Unlike traditional prefix-based context caching (CC) systems that can only reuse KV caches when the prefix is an exact match, INFOBLEND supports partial and flexible reuse of multimodal caches, even when text and image inputs are interleaved or reordered. The system tackles both system-level and algorithm-level challenges by storing large KV caches on disk and loading them in parallel, introducing a selective attention mechanism that selectively recomputes critical tokens to avoid accuracy degradation, and achieving significant improvements in response time (TTFT) and throughput with minimal quality loss. The approach is validated through experiments on two MLLMs (LLaVA 1.6 Vicuna-7B and Mistral-7B) across MMDU and SparklesEval datasets.", "questions": "N/A", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths:\n\n1. It proposes a hybrid reuse-recompute strategy based on attention sparsity and token impact, which is insightful and well-motivated.\n\n2. It demonstrates a 54% reduction in TTFT and 2× throughput over prefix caching with only minor quality loss (~13.6%).\n\n3. The paper thoughtfully combines systems design (e.g., parallel KV cache loading) and algorithmic insight (e.g., selective attention) for real-world deployment.\n\n### Weaknesses:\n\n1. The experiments are conducted only on 7B-parameter MLLMs. While understandable given hardware constraints (single H800 GPU), it limits the generalizability to large-scale production settings.\n\n2. There is no formal analysis of the quality degradation due to partial reuse or how selective attention scales with number of modalities and sequence length.\n\n3. The token selection process (based on K-distance and attention) may add computational complexity or engineering burden in practice.", "was_revised": true, "final_justification": "All concerns have been resolved.", "rebuttal": "Thank you for taking the time to review our submission.\n\nWe believe there may have been a mix-up, as your review appears to describe a paper titled INFOBLEND, which focuses on KV-cache reuse for multimodal LLMs, a topic that is entirely unrelated to our submission.\n\nWe kindly ask you to verify whether this review may have been mistakenly attached to our submission.\n\nThank you again for your time and understanding."}, {"review_id": "0bsB3toNpR", "reviewer": "Reviewer_WpXj", "summary": "The paper proposes to incorporate a new term into the LLM distillation objective. The new term is based on Gromov-Wasserstein distance that aims to preserve relations between top-k logits for each token in a sequence. Empirical results demonstrate improvement across several benchmarks.", "questions": "See weaknesses.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- I found the idea of incorporating GW into distillation interesting. Proposed GW approximation also makes it practical.\n- Empirical results overall seem positive and the ablation studies are fairly detailed.\n\nWeaknesses:\n- I'd appreciate a more detailed analysis of the GW approximation. Perhaps with a simple simulation study comparing it to other (less efficient) approaches to computing GW.\n- Ablation study in Table 4 suggests that gains due to fusing more than 2 models are fairly marginal. Main results (Table 2) also demonstrate that the fused model outperforms all of the source models in only about half of the tasks. While I acknowledge that empirical results are good overall, the (ideal) goal of a fusion method is not only to improve on average, but to also consistently outperform source models and benefit from additional source models.\n- Why did you choose k=30 for ablation studies if the optimal k=10? Does it mean that in Table 4 results for singleton source models are with k=30 and InfiGFusion results are with k=10?\n- The paper repeatedly states that the proposed GW term captures \"internal reasoning\". I don't think this claim is supported in the paper. While the GW term seems useful, how is it related to \"internal reasoning\"?", "was_revised": true, "final_justification": "Authors addressed my questions in their rebuttal. I think the method is interesting and the empirical results are solid.", "rebuttal": "Thank you for your patient review. Our responses to each point are as follows:\n\nW1: We have added an additional comprehensive simulation as Appendix I in the revised manuscripts. To verify the generality of our GLD approximation, we conduct experiments in the traditional knowledge distillation setting, where a compact student model learns from a single larger teacher model. Due to the O(N4) complexity of exact GW computation, we conducted an ablation study on 7B (Qwen 2.5) and sub-1B models (GPT2-120M, OPT-350m, Bloomz-560m) with LLama3 as the teacher, comparing generally sinkhorn computied GW and our GLD.\n\n| Teacher | Model         | Methods | Times/h | Dolly         | SelfInst      | VicunaEval    | S-NI         | Average       |\n|---------|---------------|---------|---------|---------------|---------------|---------------|--------------|---------------|\n| Teacher | Qwen 2.5-7B   | SFT     | -       | 28.71±0.31    | 26.01±0.43    | 21.32±0.48    | 42.56±0.32   | 29.65±0.39    |\n|         | LLama3        | SFT     | -       | 32.05±0.38    | 25.04±0.71    | 20.68±0.47    | 40.73±0.29   | 29.63±0.46    |\n|         | Mistral       | SFT     | -       | 31.46±0.34    | 25.15±0.73    | 20.57±0.29    | 37.56±0.40   | 28.69±0.44    |\n| Qwen 2.5| GPT2-120M     | GW      | 4.6     | 21.68±0.20    | 11.40±0.15    | 15.75±0.10    | 22.40±0.15   | 17.81±0.15    |\n|         |               | GLD     | 3.8     | 21.38±0.38    | 11.34±0.42    | 15.73±0.15    | 22.38±0.30   | 17.58±0.31    |\n|         | OPT-350m      | GW      | 6.1    | 23.10±0.15    | 12.70±0.20    | 15.80±0.15    | 22.10±0.10   | 18.43±0.15    |\n|         |               | GLD     | 5.0     | 22.80±0.48    | 12.61±0.54    | 15.77±0.67    | 21.85±0.33   | 18.26±0.51    |\n|         | Bloomz-560m   | GW      | 6.7     | 22.90±0.10    | 12.60±0.10    | 14.80±0.15    | 23.40±0.10   | 18.43±0.11    |\n|         |               | GLD     | 4.9     | 22.83±0.16    | 12.52±0.14    | 14.74±0.31    | 23.19±0.13   | 18.32±0.19    |\n| LLama3  | GPT2-120M     | GW      | 3.3     | 21.00±0.15    | 11.00±0.10    | 15.10±0.10    | 20.20±0.10   | 16.83±0.11    |\n|         |               | GLD     | 2.8     | 20.97±0.28    | 10.81±0.55    | 14.97±0.37    | 20.09±0.24   | 16.69±0.36    |\n|         | OPT-350m      | GW      | 5.9     | 22.10±0.10    | 11.00±0.15    | 15.00±0.10    | 19.50±0.10   | 16.90±0.11    |\n|         |               | GLD     | 4.4     | 22.01±0.31    | 10.97±0.33    | 14.87±0.46    | 19.21±0.25   | 16.77±0.34    |\n|         | Bloomz-560m   | GW      | 6.0     | 21.20±0.10    | 11.00±0.10    | 13.50±0.10    | 21.70±0.10   | 16.85±0.10    |\n|         |               | GLD     | 5.3     | 21.07±0.47    | 10.82±0.17    | 13.41±0.29    | 21.58±0.24   | 16.72±0.29    |\n\n(1) Performance: GW outperforms GLD (e.g., 17.81% vs. 17.58% average for Qwen 2.5-GPT2-120M) with lower variance (0.11–0.15 vs. 0.19–0.51), reflecting exact alignment.\n(2) Computational Cost: GW requires 3.3–6.7 hours vs. GLD’s 2.8–5.3 hours, due to O(N4) vs. O(NlogN)complexity.\n(3) Efficiency Trade-off: GLD achieves near-equivalent performance (e.g., 18.26% vs. 18.43% for Qwen 2.5-OPT-350m) with 15–30% less time, ensuring scalability.\nDetailed Results and Comprehensive data and comparisons will be provided in Appendix I in the final version.\n\nW2: We agree that ideally, a fusion method should not only improve average performance but also consistently outperform all source models and leverage additional sources effectively. However, the focus of this work is to introduce and evaluate the GLD formulation itself, **source re-weighting and selection** strategies is another complementary research direction, we plan to do research about it in the future work. While our results already demonstrate competitive performance across diverse tasks, we acknowledge that further robustness can be achieved through **source re-weighting and selection strategies** that address factual conflicts and model redundancy. In fact, our GLD without this strategy is compatible with existing source weighting or selection methods [1], and we view their integration as a promising future direction. \n\nW3: All results in Table 4, including singleton source models and InfiGFusion variants, are evaluated under the same setting of top-k = 30 to ensure a fair comparison. While the best top-k in Table 2 is tuned per method (e.g., k = 10 for best final performance), we intentionally choose k = 30 for ablation because at smaller k, the only ULD component struggles to retain sufficient semantic information, especially on complex reasoning tasks. Using k = 30 ensures that all model variants are provided with enough information to showcase the relative contribution of each component.\n\nW4: Thank you for highlighting this concern. We acknowledge that the term “internal reasoning” may be ambiguous and will revise the manuscript to clarify our intent. Specifically, our use of “internal reasoning” refers to the relational structure among semantic dimensions in the logit space, which often reflects intermediate concept alignment or multi-step inference patterns. The proposed GW term models these structures by aligning the graph of inter-logit relationships across models via Gromov-Wasserstein distance, rather than treating logits as independent vectors. While this does not model reasoning steps explicitly, it provides a structural inductive bias that benefits reasoning-intensive tasks. We will revise the manuscript to more precisely describe this as capturing relational patterns indicative of reasoning behavior, rather than reasoning itself.\n\n[1] Wan, Fanqi, et al. \"Knowledge Fusion of Large Language Models.\" The Twelfth International Conference on Learning Representations."}, {"review_id": "yEsp1JlugK", "reviewer": "Reviewer_k6zY", "summary": "The authors propose a new model fusion method via knowledge distillation (KD) where the inner relationship between tokens are taken into consideration by adding a distance on the token probability space as additional KD loss. Using an expensive Gromov-Wasserstein distance to measure the internal structure similarity on the token probability space, the authors further propose an approximation to significantly reduce the computation complexity. The experiments show that the methods successfully fuse different capacity from different source models and outperformed than baseline fusion methods.", "questions": "See above.\n\n1. The relative error analysis and empirical error analysis.\n2. Is it the case that without the GW distance approximation the methods can learn better?\n3. More case study based on the learned token relation instead of the output token sequence.\n\nIf at least two questions have been answered with an explicit answer, I will raise my score to 6.", "limitations": "I do not find the discussion about the limitation. I suggest the authors should add a limitation section.", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n1. The idea to use the relation between the tokens to further align the models is very impressive. The authors keenly discovered the structural relationship implicit in the output token probabilities and used this relationship to model the properties of a certain task. I believe the motivation is well-explained.\n2. The approximation seems useful. And the key insight \"the structure of each similarity matrix can be summarized using scalar node-level features\" is crucial to make the expensive distance computable.\n3. The experiments successfully proof the effectiveness of the methods. The results support the authors' claim well. The datasets contain many different fields.\n4. The paper is well-organized and reader friendly.\n\nWeaknesses:\nI do not find major weaknesses but I have some minor suggestions about the paper.\n1. Although the related work is well explained. I believe the paper [34] (or maybe other paper which first achieve cross tokenizer fusion)  should have been introduced earlier in the text to allow readers to better appreciate its innovations and avoid potential misunderstandings during reading.\n2. Besides the absolute error bound, the approximation analysis should also include a relative error analysis. Specifically, when the error bound is about O(1/N), what is generally the scale of the distance? The relative error is the key to determining whether the approximation is suitable. And I believe it is better to show some empirical results on the approximation error itself. At the same time, I understand the original O(N^4) is very expensive but I really wonder what is the performance without the approximation. I wonder whether an ablation study here is possible, (maybe just for explain the upper bound of the performance?).\n3. The current case study is based on the output sequence, which could be influenced by so many factors. I believe a more straight-forward case study (and also I are really curious about) is the actual learned tokens relation for different models and tasks. For example, can the distance really help models to learn the period of digit 1-9? Or whether some real-world concept like \"cities\" can be modeled by your methods? I really wonder whether there is some results about the interpretability of the models.", "was_revised": true, "final_justification": "strong accept.", "rebuttal": "Thank you for your careful feedback and valuable suggestions. Our responses to each point are as follows:\n\nW1: Thanks for your suggestion! We agree that introducing the concept of cross-tokenizer fusion, as exemplified by [34] , earlier in the paper can enhance clarity and better contextualize InfiFusion’s innovations. In the revised manuscript, we incorporate a brief mention of cross-tokenizer fusion in Section 1 to highlight its relevance to model fusion and to clarify how cross-tokenizer distillation builds upon and extends this paradigm. Specifically, we will note that while [34] introduced a universal logit distillation loss for aligning models with different tokenizers, InfiGFusion advances this by capturing semantic dependencies through graph-based alignment, addressing limitations in token-level methods for complex reasoning tasks.\nW2: For Scale of GW Distance and Relative Error Analysis, we analyzed the scale of the GW distance in our experiments, 14B model and 130K data, using datasets like BBH and MMLU (supplement in Appendix F.1 & F.2 ) and make additional experiments with other approximation algorithm. Due to the complexity, it`s hard to directly calculate the original GW distance.\n| **BBH Subset**            | **Logical Deduction** | **Multistep Arithmetic** | **Causal Judgement**  | **Dyck Languages** |\n|---------------------------|-----------------------|--------------------------|----------------------|-------------------------|\n| **Our Approximation (Sorting-Based, O(N log N))** | | | | | |\n| GW Distance               | 0.0356              | 0.0614                 | 0.0445             | 0.0210           |\n| Absolute Error Bound          | 0.00015            | 0.00015                |0.00015             | 0.00015           |\n| Relative Error (%)        | 0.421             | 0.244               | 0，337             | 0.714         |\n Performance (%)           | 89.2             | 99.6              | 70.0            | 94.4               | 40.0          |\n| **Sinkhorn GW (O(N² log N))** | | | | | | \n| GW Distance | 0.02145 | 0.0618 | 0.0449 | 0.0213 | \n| Absolute Error Bound | 0.00009 | 0.00009 | 0.00009 | 0.00009 | \n| Relative Error (%) | 0.419 | 0.145 | 0.200 | 0.422 | \n| Performance (%)           | 89.90         | 99.2          | 70.4        | 94.1           | 40.7     |\n| **Entropic GW (O(N² log N))** | | | | | | \n| GW Distance | 0.0314 | 0.0617 | 0.0448| 0.0212 | \n| Absolute Error | 0.00012 | 0.00012 | 0.00012  | 0.00012 | \n| Relative Error (%) | 0.382 | 0.194 | 0.267 | 0.566 | \n| Performance (%)           | 89.7             | 99.6                | 70.7           | 93.9               | 40.0          |\n\n\nThe relative error analysis is included, computed as the absolute error bound divided by the GW distance (absolute errors are calculated following the original paper [1][2]). The GW distance scale ranges from 0.0210 to 0.0614 across BBH subsets, with the absolute error bound fixed at 0.00015, resulting in relative errors of 0.244% to 0.714%. This indicates suitability, as errors remain below 1%, decreasing with larger GW distances. Empirical results show Sorting-Based’s absolute error aligns with the O(1/N) bound (n=150k and m=100k), while Sinkhorn and Entropic GW offer tighter approximations, validated by performance gains.\n\nFor the upper bound of the performance, due to this cost, we conducted ablation studies on 7B (Qwen 2.5) and sub-1B models (GPT2-120M, OPT-350m, Bloomz-560m) with LLama3 as the teacher. Results show GW consistently achieves higher or equal performance than GLD (e.g., Qwen 2.5-GPT2-120M: 17.81% vs. 17.58% average), with smaller variance (e.g., 0.15 vs. 0.31), indicating better alignment stability. The upper bound performance gain is modest (e.g., 0.23–0.31% on average), suggesting GLD’s O(NlogN) efficiency retains most benefits. Computational time for GW (2.6–4.7h) exceeds GLD (2.8–5.3h), confirming the trade-off, making GLD suitable for large-scale applications.\n| Teacher | Model         | Methods | Times/h | Dolly         | SelfInst      | VicunaEval    | S-NI         | Average       |\n|---------|---------------|---------|---------|---------------|---------------|---------------|--------------|---------------|\n| Teacher | Qwen 2.5-7B   | SFT     | -       | 28.71±0.31    | 26.01±0.43    | 21.32±0.48    | 42.56±0.32   | 29.65±0.39    |\n|         | LLama3        | SFT     | -       | 32.05±0.38    | 25.04±0.71    | 20.68±0.47    | 40.73±0.29   | 29.63±0.46    |\n|         | Mistral       | SFT     | -       | 31.46±0.34    | 25.15±0.73    | 20.57±0.29    | 37.56±0.40   | 28.69±0.44    |\n| Qwen 2.5| GPT2-120M     | GW      | 4.6     | 21.68±0.20    | 11.40±0.15    | 15.75±0.10    | 22.40±0.15   | 17.81±0.15    |\n|         |               | GLD     | 3.8     | 21.38±0.38    | 11.34±0.42    | 15.73±0.15    | 22.38±0.30   | 17.58±0.31    |\n|         | OPT-350m      | GW      | 6.1    | 23.10±0.15    | 12.70±0.20    | 15.80±0.15    | 22.10±0.10   | 18.43±0.15    |\n|         |               | GLD     | 5.0     | 22.80±0.48    | 12.61±0.54    | 15.77±0.67    | 21.85±0.33   | 18.26±0.51    |\n|         | Bloomz-560m   | GW      | 6.7     | 22.90±0.10    | 12.60±0.10    | 14.80±0.15    | 23.40±0.10   | 18.43±0.11    |\n|         |               | GLD     | 4.9     | 22.83±0.16    | 12.52±0.14    | 14.74±0.31    | 23.19±0.13   | 18.32±0.19    |\n| LLama3  | GPT2-120M     | GW      | 3.3     | 21.00±0.15    | 11.00±0.10    | 15.10±0.10    | 20.20±0.10   | 16.83±0.11    |\n|         |               | GLD     | 2.8     | 20.97±0.28    | 10.81±0.55    | 14.97±0.37    | 20.09±0.24   | 16.69±0.36    |\n|         | OPT-350m      | GW      | 5.9     | 22.10±0.10    | 11.00±0.15    | 15.00±0.10    | 19.50±0.10   | 16.90±0.11    |\n|         |               | GLD     | 4.4     | 22.01±0.31    | 10.97±0.33    | 14.87±0.46    | 19.21±0.25   | 16.77±0.34    |\n|         | Bloomz-560m   | GW      | 6.0     | 21.20±0.10    | 11.00±0.10    | 13.50±0.10    | 21.70±0.10   | 16.85±0.10    |\n|         |               | GLD     | 5.3     | 21.07±0.47    | 10.82±0.17    | 13.41±0.29    | 21.58±0.24   | 16.72±0.29    |\n\n\nW3: It’s a really insightful point! We make additional analysis, for real-world concepts like “cities,” GLD’s feature-wise graph captures token co-activation patterns, enabling relational alignment across models. We analyzed a sample sentence from the Causal Judgement task: “New York and Boston are closer than Los Angeles and Seattle.” and plot the token distribution of GLD and without GLD. Figure without GLD shows the token distribution, with higher activation weights for “New York” and “Boston” due to proximity, while Figure with GLD reveals stronger edge connections between these tokens in the learned graph, reflecting semantic similarity, unlike token-level baselines, which overlook these relations.\nTo enhance interpretability, we will add these case study and figures in Appendix F using this example, demonstrating how GW distance aligns these relations, and compare it to a baseline.\n\n[1] Scetbon, Meyer, Gabriel Peyré, and Marco Cuturi. \"Linear-time gromov wasserstein distances using low rank couplings and costs.\" International Conference on Machine Learning. PMLR, 2022.\n[2] Rioux, Gabriel, Ziv Goldfeld, and Kengo Kato. \"Entropic gromov-wasserstein distances: Stability and algorithms.\" Journal of Machine Learning Research 25.363 (2024): 1-52."}, {"review_id": "Agj2KRoIBX", "reviewer": "Reviewer_wyJ9", "summary": "This paper introduces InfiGFusion, a novel framework for fusing heterogeneous large language models (LLMs) that overcomes the limitations of prior logit-based methods. The authors argue that existing techniques treat vocabulary dimensions independently, failing to capture the crucial semantic dependencies and co-activation patterns that underlie a model's reasoning process. To address this, InfiGFusion proposes a structure-aware approach centered on a new Graph-on-Logits Distillation (GLD) loss. This method constructs feature-level graphs from the logits, where nodes represent vocabulary dimensions and edges quantify their joint activations, thereby capturing the model's internal reasoning structure. To make this approach computationally feasible, the authors develop a novel, sorting-based closed-form approximation of the Gromov-Wasserstein (GW) distance, reducing its complexity from a prohibitive \n$O(n^4)$ to a highly efficient $O(n\\log n)$ with provable approximation guarantees. Experiments across 11 benchmarks demonstrate that InfiGFusion significantly outperforms existing fusion methods and baselines, showing particular strength in complex reasoning, mathematics, and coding tasks.", "questions": "Could the authors please provide error bars or significance tests for a critical subset of the results? I would suggest focusing on:\n\n- The main average scores in Table 2, comparing InfiGFusion to the Pivot-SFT and the next-best fusion baseline.\n\n- The most dramatic gains on complex reasoning tasks reported in Table 3, specifically \"Multistep Arithmetic\" and \"Causal Judgement,\" which are central to the paper's narrative.", "limitations": "please refer to the weaknesses.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n- Existing methods that align output logits often treat vocabulary dimensions independently, ignoring the rich semantic relationships and dependencies that underpin a model's reasoning process. InfiGFusion is the first framework to propose a structure-aware approach, representing logit co-activation patterns as graphs and aligning these structures. This is a fundamentally new and promising direction for model fusion.\n\n- The method shows dramatic improvements on tasks that require multi-step and relational inference, such as a +35.6 point gain on Multistep Arithmetic and a +37.06 point gain on Causal Judgement over the SFT baseline. This provides strong evidence that capturing semantic structure is crucial for fusing complex reasoning abilities.\n\n-  A major challenge of using the Gromov-Wasserstein (GW) distance is its prohibitive computational complexity ($O(n^4)$). A key contribution of this work is a novel sorting-based approximation that reduces the complexity to a practical  $O(n\\ log n)$. Crucially, the authors provide a  provable error bound for this approximation, ensuring its fidelity to the original metric.\n\n## Weaknesses\n\n- The authors explicitly state in the NeurIPS checklist that they did not report error bars or conduct statistical significance tests, citing computational cost. While many of the reported gains are substantial, this is a notable omission for an empirical paper. Without statistical validation, it is difficult to be certain that smaller performance improvements are meaningful and not simply the result of experimental noise.\n\n- The paper acknowledges that the method focuses on aligning relational consistency and lacks an explicit mechanism for resolving factual conflicts between source models. This is a significant limitation in practice, as fusing models with contradictory knowledge could lead to unpredictable or factually incorrect, though structurally coherent, outputs.\n\n- The authors note that the benefits of GLD are marginal for tasks dominated by factual recall, where simple token matching is sufficient. While this is a reasonable trade-off, a more in-depth discussion of when structural alignment might be unnecessary or even detrimental would strengthen the paper.", "was_revised": true, "final_justification": "All of my concerns are resolved during my rebuttal. After looking into other reviewers' comment, I raise my score and confidence.", "rebuttal": "Thanks for your insightful feedback. Our responses to each point are as follows:\n\nW1 & Q1: Thank you for highlighting the importance of statistical validation. While we follow the previous works experimental design like FuseLLM and FuseChat, we acknowledge that reporting confidence intervals or statistical tests would strengthen the empirical evidence. Importantly, many of the gains we observed (e.g., +35.6 on Multistep Arithmetic and 37.06 on Causal Judgement) are substantial and consistent across datasets. It can make sure that the results are not noise.\n\nWe also make additional experiments with 4 different seed settings, which provides the variance and interval:\n\n| Models      | Math          |       |       | Coding   | General Reasoning |       |       |       | Instruct. | Text Reasoning |       | Avg         | Model Size | GPU Hour |\n|-------------|---------------|-------|-------|----------|-------------------|-------|-------|-------|-----------|----------------|-------|-------------|------------|----------|\n|             | GSM8K         | MATH  | ThQA  | MBPP     | HEval             | BBH   | ARC   | MMLU  | IFEval    | DROP            | HS    |             |            |          |\n| SFT         | 90.02±0.08    | 82.46±0.5 | 54.71±0.31 | 75.33±2.53 | 86.20±0.61 | 66.85±0.38 | 93.90±0.34 | 85.76±0.45 | 77.68±0.33 | 89.34±0.11 | 87.64±0.12 | 81.01 ± 0.46 | 14B      | 120        |\n| InfiFusion  | 90.12±0.11    | 80.91±0.16 | 54.64±0.33 | 79.61±1.92 | 84.74±0.91 | 80.92±0.63 | 94.21±0.39 | 85.79±0.38 | 76.04±0.82 | 89.29±0.15 | 87.93±0.12 | 82.01±0.42 | 14B      | 160        |\n| InfiGFusion | 90.46±0.06    | 81.76±0.16 | 54.99±0.38 | 84.66±1.55 | 86.00±0.91 | 86.25±0.88 | 94.49±0.45 | 84.93±0.31 | 80.28±0.71 | 89.86±0.19 | 88.34±0.13 | 83.74 ± 0.13 | 14B      | 195        |\n\nTo demonstrate the effectiveness and generalization of our GLD, we also provide additional experiments on the small size distillation with variance and interval:\n\n| Teacher       | Model         | Methods | Times/h | Dolly      | Selflnst  | VicunaEval | S-NI     | Average   | Improv. |\n|---------------|---------------|---------|---------|------------|-----------|------------|----------|-----------|---------|\n|               | Qwen 2.5-7B   | SFT     |         | 28.71±0.31 | 26.01±0.43 | 21.32±0.48 | 42.56±0.32 | 29.65±0.39 |         |\n|               | LLama3        | SFT     |         | 32.05±0.38 | 25.04±0.71 | 20.68±0.47 | 40.73±0.29 | 29.63±0.46 |         |\n|               | Mistral       | SFT     |         | 31.46±0.34 | 25.15±0.73 | 20.57±0.29 | 37.56±0.40 | 28.69±0.44 |         |\n| Qwen 2.5      | GPT2-120M     | MinED   | 6.5     | 20.55±0.15 | 10.88±0.36 | 14.90±0.63 | 22.12±0.36 | 17.11±0.38 | 3.48%   |\n|               |               | ULD     | 2.5     | 20.88±0.56 | 11.02±0.33 | 15.45±0.36 | 21.24±0.37 | 17.15±0.41 | 3.27%   |\n|               |               | GLD     | 3.8     | 21.38±0.38 | **11.34±0.42** | **15.73±0.15** | **22.38±0.30** | **17.58±0.31** | -       |\n|               | OPT-350m      | MinED   | 5.0     | 22.62±0.34 | 12.37±0.45 | 15.52±0.30 | 21.99±0.12 | 18.13±0.30 | 0.73%   |\n|               |               | ULD     | 4.5     | 22.36±0.29 | 11.38±0.50 | 14.61±0.36 | 21.80±0.59 | 17.54±0.44 | 4.11%   |\n|               |               | GLD     | 5.0     | 22.80±0.48 | **12.61±0.54** | **15.77±0.67** | 21.85±0.33 | **18.26±0.51** | -       |\n|               | Bloomz-560m   | MinED   | 4.2     | **22.86±0.56** | 11.78±0.34 | **14.82±0.39** | 22.83±0.30 | 18.07±0.40 | 1.37%   |\n|               |               | ULD     | 5.2     | 22.51±0.32 | 12.48±0.44 | 14.08±0.45 | 22.82±0.21 | 17.97±0.36 | 1.93%   |\n|               |               | GLD     | 4.9     | 22.83±0.16 | **12.52±0.14** | 14.74±0.31 | 23.19±0.13 | **18.32±0.19** | -       |\n| LLama3        | GPT2-120M     | MinED   | 2.9     | 19.93±0.36 | 10.15±0.24 | 14.38±0.43 | 19.78±0.23 | 16.06±0.32 | 3.44%   |\n|               |               | ULD     | 2.5     | 19.66±0.49 | 9.84±0.31  | 14.59±0.50 | 19.77±0.14 | 15.97±0.36 | 4.06%   |\n|               |               | GLD     | 2.8     | **20.97±0.28** | **10.81±0.55** | 14.97±0.37 | **20.09±0.24** | **16.69±0.36** | -       |\n|               | OPT-350m      | MinED   | 5.3     | 21.25±0.27 | 10.06±0.38 | **15.28±0.41** | 19.04±0.32 | 16.41±0.35 | 2.18%   |\n|               |               | ULD     | 4.4     | 21.08±0.56 | 9.85±0.42  | 14.89±0.50 | 19.13±0.58 | 16.24±0.52 | 3.25%   |\n|               |               | GLD     | 4.7     | **22.01±0.31** | **10.97±0.33** | 14.87±0.46 | 19.21±0.25 | **16.77±0.34** | -       |\n|               | Bloomz-560m   | MinED   | 4.7     | 20.16±0.21 | 10.81±0.16 | 13.36±0.44 | 20.62±0.17 | 16.24±0.25 | 2.97%   |\n|               |               | ULD     | 5.3     | 20.69±0.49 | 10.50±0.64 | 12.97±0.61 | 20.71±0.18 | 16.22±0.48 | 3.10%   |\n|               |               | GLD     | 5.5     | **21.07±0.47** | 10.82±0.17 | 13.41±0.29 | **21.58±0.24** | **16.72±0.29** | -       |\n| Mistral       | GPT2-120M     | MinED   | 2.9     | 20.47±0.62 | 10.52±0.79 | 15.36±0.27 | 20.59±0.54 | 16.74±0.56 | 8.34%   |\n|               |               | ULD     | 2.6     | 20.00±0.52 | 10.53±0.81 | 15.29±0.20 | 20.57±0.16 | 16.60±0.42 | 9.23%   |\n|               |               | GLD     | 2.6     | **21.90±0.31** | **11.89±0.41** | 15.99±0.31 | **22.74±0.27** | **18.13±0.33** | -       |\n|               | OPT-350m      | MinED   | 3.1     | 22.67±0.29 | 11.96±0.74 | 15.85±0.65 | 22.55±0.20 | 18.26±0.47 | 3.63%   |\n|               |               | ULD     | 2.3     | 22.45±0.38 | 11.19±0.88 | 15.41±0.60 | 23.17±0.32 | 18.06±0.55 | 4.79%   |\n|               |               | GLD     | 2.8     | **23.72±0.36** | **12.73±0.30** | 15.14±0.53 | **24.11±0.28** | **18.92±0.36** | -       |\n|               | Bloomz-560m   | MinED   | 3.9     | 22.73±0.18 | 12.30±0.37 | 15.58±0.16 | 23.97±0.13 | 18.65±0.21 | 2.49%   |\n|               |               | ULD     | 3.4     | 22.21±0.40 | 12.23±0.55 | 15.11±0.39 | 23.64±0.37 | 18.30±0.43 | 4.44%   |\n|               |               | GLD     | 3.7     | 23.22±0.38 | **12.87±0.49** | **15.92±0.29** | **24.43±0.11** | **19.11±0.32** | -       |\n\nW2: That`s a good point! We agree that InfiGFusion have the problem of resolving factual conflicts between source models. **However, the source re-weighting and selection strategy, which aims to resolve the factual conflicts between source models, is a complementary research direction, and our work mainly focuses on aligning semantic structure and design a better GLD loss.** \nIn fact, our framework can be naturally extended by integrating factual confidence estimation or external verifiers. We have tested some existing factual conflicts resolving strategy, but the results is not significant. We thank you for pointing out this practical limitation and will discuss this more thoroughly in the final version. Designing a better factual conflicts resolution is viewed as one of our future works in our plan. But it is not the concerns in this work.\nW3: We appreciate the reviewer’s insightful observation. Due to space limitations, our discussion on when structural alignment is less impactful was brief in the main text. However, Appendices F–I provide a detailed analysis. We offers a per-task performance breakdown over 84 benchmarks (BBH + MMLU), clearly showing that GLD brings the most gains in reasoning-intensive tasks like Multistep Arithmetic, Causal Judgement, while maintaining parity in fact-centric domains, such as Biology, US History in Appendix F. Appendix G includes qualitative case studies illustrating how GLD improves multi-hop and dependency-based reasoning, while not harming performance on fact-centric prompts. Appendix H reports robustness trends and failure cases, confirming that GLD is beneficial or neutral—but not detrimental—even in tasks where token identity dominates.\nIn our updated version, we will include this more dedicated subsection discussing this, and refer readers to detailed case studies and task-specific analyses in the appendix:\nOur qualitative analysis (Section 4.6) further illustrates this: InfiGFusion provides nuanced causal disambiguation by modeling reasoning steps (e.g., intent → misfire → injury, or criminal motive → assassination → explosion), while SFT models tend to respond with shallow pattern-based answers. In contrast, on factual recall tasks such as *International Law*, *Ruin Names*, or *US History*, where precise token identity or entity retrieval dominates, GLD offers limited gains (+0–1%). However, importantly, we observe that InfiGFusion does not degrade performance on such tasks. For example, on MMLU factual domains (e.g., *Biology*, *Psychology*), InfiGFusion matches SFT within ±1.5%, indicating that our method is robust even when structural signals are not essential.\n\nQ2: \nThank you for the insightful question. The gains on Multistep Arithmetic and Causal Judgement are consistent and not isolated. As detailed in Section 4.3 and Appendix F, InfiGFusion significantly outperforms both individual sources and strong fusion baselines across multiple structure-heavy BBH tasks. Expect for Multistep Arithmetic and Causal Judgement, additional consistent gains are observed on Formal Fallacies, Boolean Expressions, and Word Sorting. It is the overall results across different scenarios that demonstrates the general significant abilities in complex reasoning."}]

Author rebuttal:

Meta-review: InfiGFusion proposes a structure-aware fusion method that augments KD with a Graph-on-Logits Distillation term, aligning token co-activation graphs via a GW-style objective and a sorting-based O(n log n) approximation. The motivation is clear and the idea is novel for fusion. Empirically, results across 11 benchmarks are solid, with the most convincing gains on structure-heavy reasoning (e.g., Multistep Arithmetic, Causal Judgement). The rebuttal strengthened the paper with variance/CIs, approximation studies, clarified top-k choices, and toned down claims about “internal reasoning” to “relational patterns.” Limitations remain: improvements are not universal (fused not better than best single source everywhere), gains from >2 sources are modest, the method does not resolve factual conflicts among sources, and approximation analysis is split across places and should be unified. A misassigned review was corrected; other presentation wrinkles are manageable for camera-ready. Overall: a novel, practical contribution with clear upside on reasoning tasks, I recommend as poster; authors please polish the unified GW-approx analysis, top-k narrative, and a couple of interpretability figures in the final version.
Decision: Accept (poster)