Exploring Model Kinship for Merging Large Language Models

## Abstract

Abstract Model merging has become one of the key technologies for enhancing the capabilities and efficiency of Large Language Models (LLMs). However, our understanding of the expected performance gains and principles when merging any two models remains limited. In this work, we introduce model kinship , the degree of similarity or relatedness between LLMs, analogous to biological evolution . With comprehensive empirical analysis, we find that there is a certain relationship between model kinship and the performance gains after model merging, which can help guide our selection of candidate models. Inspired by this, we propose a new model merging strategy: Top- k 𝑘 k italic_k Greedy Merging with Model Kinship, which can yield better performance on benchmark datasets. Specifically, we discover that using model kinship as a criterion can assist us in continuously performing model merging, alleviating the degradation (local optima) in model evolution, whereas model kinship can serve as a guide to escape these traps 1 1 1 Code is available at https://github.com/zjunlp/ModelKinship . .

## 1 Introduction

Fine-tuning pre-trained models (PTMs) for downstream tasks has become a popular practice, particularly demonstrating significant effectiveness in Large Language Models (LLMs) (Kolesnikov et al., ; Qiu et al., ; Askell et al., ; Ouyang et al., ; Zhao et al., ).
However, deploying separate fine-tuned models for each task can be resource-intensive (Fifty et al., ), which drives the increasing demand for multitask learning solutions (Zhang & Yang, ; Lu et al., ; Liu et al., ).
Recent studies suggest that model merging (Singh & Jaggi, ; Sung et al., ; Goddard et al., ; Matena & Raffel, ; Yang et al., ) offers a viable approach for achieving multitask objectives by integrating multiple expert models.
Furthermore, advancements in model merging toolkits (Goddard et al., ; Tang et al., ) enable users with limited expertise to easily conduct merging experiments, leading to an evolution of LLMs for the community.

To date, through model merging techniques, resercheres have developed many more powerful LLMs through iterative model merging (Beeching et al., ), and to some extent, achieved model evolution (Figure 1(c)).
Despite these successes, progress has predominantly relied on trial and error, along with extensive human expertise, but lacks formalized guidance and standardized procedures.
As the merging iterations progress, achieving further generalization gains becomes increasingly challenging (More details in Section 3).
For example, as shown in Figure 1, model merging often resembles the process of hybrid evolution in biology, where the next generation may not show significant improvements or may even regress, highlighting the imperative for a deeper exploration of the underlying mechanisms driving these advancements.

To address this, we introduce model kinship, a metric inspired by the concept of kinship (Sahlins, ) from evolutionary biology (Figure 1(a)).
This metric is designed to estimate the degree of similarity or relatedness between LLMs during the iterative model merging process, offering insights intended to enhance the effectiveness of the merging strategy.
We utilize the model kinship to conduct a comprehensive analysis of model merging experiments from two perspectives: the overall merging process, including various independent merge experiments and the evolution path of specific models, demostrating the complete merging trajectory.

Figure: Figure 1: An intuitive comparison between wheat evolution and model evolution. An interesting parallel can be drawn between biological reproduction (Part a) and the process of model evolution (Part b). In biological systems, offspring inherit genetic material from both parents, forming a new genotype through the combination of parental traits. Similarly, in model merging, the merged model inherits parameters or weights from the contributing models. Part c demonstrates the iterative execution of model evolution. Starting with a group of LLMs, the repository evolves through a Selection-Merge-Recycle iteration. To be noted, model kinship can serve as an effective tool to guide this iterative model merging process (e.g., infer whether there may be gains after model merging.).
Refer to caption: extracted/5926041/Figures/main.jpg

Model kinship correlates with average performance gain in model merging.
Emperically, we find that there is a strong correlation between variations in multitask capability, estimated by average task performance, and model kinship, which can help guide our selection of candidate models.
We also notice that the model merging process has two stages: learning, where models experience significant performance improvements and saturation, where further improvements diminish and eventually stagnate.
We think that the stagnation of improvements may be due to convergence in weight space, suggesting the presence of optimization challenges like local optima traps.

Inspired by this, we propose a new model merging strategy: Top-$k$ Greedy Merging with Model Kinship.
Specifically, we find that leveraging model kinship as a criterion enables more effective model merging, helping to mitigate degradation and avoid local optima during model evolution.
Model kinship also proves useful as an early stopping criterion, improving the efficiency of the merging process.
Overall, this paper makes three key contributions:

- 1.
Introduction of Model Kinship: We introduce model kinship, designed to assess the degree of similarity or relatedness between LLMs during the merging process, which can guide model merging strategies and holds promise for advancing auto-merging research.
- 2.
Empirical Analysis of Model Evolution: We present a comprehensive empirical analysis of model evolution through iterative merging.
Our findings highlight the dynamics of multitask performance improvement and stagnation.
Additionally, we propose a preliminary explaination of the underlying mechanisms using model kinship.
- 3.
Practical Model Merging Strategies using Model Kinship: We demostrate how model kinship guides the model merging process to tackle optimization challenges, and provide practical strategies: Top-$k$ Greedy Merging with Model Kinship, to enhance efficiency and effectiveness of model evolution.

## 2 Background

### 2.1 Model Merging

Model merging aims to integrate two or more domain-specific models into a unified framework, thereby harnessing their compositive capabilities across multiple tasks (Sung et al., ).
While this approach shares conceptual similarities with ensemble methods (Dietterich et al., ; Dong et al., ; Jiang et al., ), model merging generates a single, generalized model, avoiding the increased inference time associated with ensembles.
Let $f_{i}$ represent the $i$-th model for merging, each with its unique parameters $\theta_{i}$. If the merging process follows method $\mathcal{F}$, the prediction $\hat{y}$ of the merged model $f_{\text{merge}}$ for input $x$ is:

$$ $\hat{y}=f_{\text{merge}}(x)=\mathcal{F}\left(f_{1}(x;\theta_{1}),f_{2}(x; \theta_{2}),\ldots,f_{n}(x;\theta_{n})\right)$ (1) $$

When employing parameter averaging methods, the merged model preserves the same architecture and parameter size as the original models, enabling it to be reused in future merging processes(^2^22This paper focuses solely on model merging with identical architectures, where the merged model retains the same parameters as before the merge.).
This makes it feasible to iteratively enhance the model through repeated applications of model merging.
In this paper, we refer to this process as “Model Evolution”.
Empirical evidence on open LLM leaderboard (Beeching et al., ) demonstrates that model evolution can successfully produce highly generalized models, often outperforming their original ones (Maxime Labonne, ).

### 2.2 Model Kinship

Intuitively, understanding the task capabilities of models is crucial for effective model merging.
However, the black-box nature of LLMs presents significant challenges in comprehending their advantages Shi et al. () and task-specific capabilities.
For fine-tuned models, task benchmark results (Gao et al., ; Li et al., ) and the characteristics of the fine-tuning datasets are often used as indicators of task-related knowledge.
Yet, these metrics often fall short in comprehensive comparisons.
The internal relationships between tasks and datasets can arise confusion, particularly in complex merging scenarios that involve multiple tasks.

Motivated by the parallel drawn between biological reproduction and the process of model evolution (as seen in Figure 1), we propose that the concept of *kinship*, which is central to evolutionary biology for understanding breeding relationships and human genealogies (Thompson, ), can be conceptually extended to the field of model merging, specifically to describe the kinship between models.
To be noted, Ilharco et al. () has proposed Task Arithmetic and offered new insights into how the merging process preserves and blends task information from the original models.
(Ilharco et al., ) propose to use task vectors, defined as the differences in weight space between fine-tuned models and a pre-trained model, and demonstrate that arithmetic operations on these task vectors can serve as an effective method for model merging.

Inspired by this idea, we introduce the concept of Model Kinship, a metric designed to assess the degree of similarity or relatedness between large language models (LLMs) based on their “genetic” information (a.k.a. the changes in weights during model evolution).
Considering two target models involved in a single merge, the weights of each target model are denoted as $\theta_{\text{target}}\in\mathbb{R}^{d}$.
In this paper, it is significant to claim that a target model may either be a fine-tuned model or a previously merged model.
Similarly, $\theta_{\text{base}}\in\mathbb{R}^{d}$ represents the weights of the base model from which the two target models are derived.
The difference of weights $\delta_{\text{target}}$ between the target model and the base model is denoted as:

$$ $\delta_{\text{target}}=\theta_{\text{target}}-\theta_{\text{base}}.$ $$

For a model merging process where models have been fine-tuned or merged from the same base model, the model kinship $r$ can be defined using the similarity metric function $sim(\cdot,\cdot)$, which measures the similarity between two model weight differences.
For $n$ models, we formally define the model kinship $r$ as:

$$ $r=\frac{2}{n(n-1)}\sum_{1\leq i<j\leq n}sim(\delta_{\text{i}},\delta_{\text{j} }),$ $$

where $\delta_{\text{i}}$ and $\delta_{\text{j}}$ are the weight differences between the $i$-th and $j$-th target models and the base model, respectively.

## 3 Preliminary Analysis of Model Kinship

We begin with a preliminary analysis of model kinship with open-sourced LLMs from the community to better understand model evolution.

### 3.1 Evaluation Metrics

Let $T$ be the set of tasks in the task group, where $T=\{T_{1},T_{2},\ldots,T_{n}\}$.
Each task $T_{i}$ in the set $T$ is associated with a performance measure $P_{i}$ for the LLM.
For a multitask objective, the Average Task Performance (Avg.) $\bar{P}$ is calculated using the equation:

$$ $\bar{P}=\frac{1}{n}\sum_{i=1}^{n}P_{\text{i}}$ (2) $$

To evaluate the effectiveness of a single merge, we propose the merge gain metric.
Assume we have two models $m_{pre-1}$ and $m_{pre-2}$ and their average task performance are $\bar{P}_{pre-1}$ and $\bar{P}_{pre-2}$, intuitively, we believe the $\bar{P}_{\text{merged}}$ lie around the mean of $\bar{P}_{pre-1}$ and $\bar{P}_{pre-2}$.
The merge gain is calculated as the difference of $\bar{P}_{\text{merged}}$ from the mean value of $\bar{P}_{pre-1}$ and $\bar{P}_{pre-2}$.
For a merging recipe with $k$ models, the merge gain is:

$$ $Gain=\bar{P}_{\text{merged}}-\frac{1}{k}\sum_{i=1}^{k}\bar{P}_{\text{pre-i}}$ (3) $$

In the following analysis, we use the task group T = {ARC, HellaSwag, MMLU, TruthfulQA, Winogrande, GSM8K}.
All models are either fine-tuned or merged from the Mistral-7B architecture.

### 3.2 Correlation Analysis of Model Kinship and Performance Gain

**Table 1: Correlation of Model Kinship based on different correlation function $sim(\cdot,\cdot)$ with Merge Gain, along with their corresponding p-values.**
| Metric | Correlation | Correlation |
| --- | --- | --- |
|  | (Normal Value) | (Absolute Value) |
| PCC | -0.50 | -0.59 |
| P-value | 0.063 | 0.023 |
| CS | -0.45 | -0.66 |
| P-value | 0.098 | 0.008 |
| ED | 0.46 | 0.67 |
| P-value | 0.091 | 0.007 |

In this analysis, we examine the distribution of merge gain and model kinship based on Pearson Correlation Coefficient (PCC), Cosine Similarity (CS) and Euclidean Distance (ED) in open-sourced LLMs, originating from the Mistral-7B (Jiang et al., ).
Those models are obtained from the HuggingFace, with assistance from the Open LLM Leaderboard (Details in Appendix A.).

#### 3.2.1 Results

Figure: Figure 2: Distribution of Sample Experiments: Relationship Between Model Kinship (X-axis) and Merge Gain (Y-axis). Model Kinships are calculated using the Pearson Correlation Coefficient (PCC), Cosine Similarity (CS) and Euclidean Distance (ED).
Refer to caption: x1.png

Figure 2 illustrates the distribution of model kinship based on three similarity metrics (PCC, CS, ED) in relation to merge gain.
The scatter plots reveal a moderate correlation between model kinship and merge gain, as indicated by the trend lines.
To further quantify these relationships, the correlation value (use Pearson Correlation Coefficient) between model kinship and merge gain are calculated, as detailed in the second column of Table 1.
While moderate correlations are observed for all three metrics, the corresponding p-values indicate a weak level of statistical significance, ranging from 0.05 to 0.1.
In contrast, when examining absolute merge gain, we find stronger and statistically significant correlations, as shown in the third column of Table 1.
These results suggest that model kinship alone is insufficient for predicting whether a model can achieve generalization gains through merging.
However, it may serve as a key factor in determining the upper limit of merge gains, thus indicating potential improvements. Since no significant differences are observed among the three metrics, we will focus solely on model kinship based on PCC in the following sections to simplify the demonstration.

### 3.3 Sequence Analysis of Model Evolution Paths

In this analysis, we examine changes in performance and model kinship across independent model evolution paths to identify the phased pattern of the merging process.
We focus on the yamshadow experiment 28-7B (Labonne, ), a Mistral 7B architecture model ranked as the top 7B merged model on the Open LLM Leaderboard. From its model family tree, we extract two primary merging paths: Path 1 and Path 2.

#### 3.3.1 Results

Figure: Figure 3: Change in Average Task Performance and Merge Gain across the Model Evolution process: The selected paths originate from two distinct initial models, with the saturation stage observed after the vertical line. Note that the generation of Path 2 is aligned with Path 1 for demonstration purposes.
Refer to caption: x2.png

Figure: Figure 4: Comparision between Model Kinship (measured by Pearson Correlation Coefficient) and Average Task Performance (normalized to the same value scale).
Refer to caption: x3.png

We first focus on the average task performance and merge gains throughout the model evolution path (Figure 3.) Detailed data and branch information are summarized in Appendix A).
Our observations indicate that the performance improvements of the iterative merging process are not linear and can be divided into two stages:

- •
Learning Stage. In this stage, the average task performance generally experiences a rapid increase. Noticeable merge gains suggest that the merged models are continually acquiring multitask capabilities through the merging process.
- •
Saturation Stage. As the process continues, improvements begin to plateau.
During this stage, the merge gains approach zero, indicating that the model can no longer benefit from the merging process and has ceased to improve.

Additionally, we compare the trend of model kinship with average task performance.
Figure 4 illustrates the changes in model kinship alongside average task performance (normalized to the same range as the corresponding metric) throughout the model evolution paths.
In addition to the notable correlation observed between average task performance and model kinship throughout the evolution process, model kinship exhibits a similar stage-specific pattern, particularly evident in the saturation stage, suggesting a potential relationship with the underlying cause of saturation.

### 3.4 Analysis of the Model Kinship in Different Merging Stages

Findings in previous analysis raises the question of “how model kinship affects performance improvements in model evolution”. To investigate the causality between model kinship and the stagnation of improvements, we examine the variation of model kinship across different merging stages from a broader perspective.

Given the community’s predominant use of the performance-prior strategy, we calculate model kinship among models with similar performance, simulating the selection of top-performing models at each stage. For this analysis, we randomly select 5 models from each merging stage - Saturation Stage ($\geq 0.75$), Learning Stage ($<$0.75 and $\geq$0.73), and Initial Merges (fine-tuned models) to form three foundation model groups, representing potential merges at different stages of model evolution.

Figure: Figure 5: The Model Kinship Matrices for the three model groups. Each element represents the model kinship value between the corresponding models. In Group B and C, the merged models are arranged by average task performance, ordered from high to low (left to right).
Refer to caption: x4.png

Figure 5 illustrates the model kinship between models within each group.
Our analysis reveals that model kinship increases with the average task performance, even across models that follow different evolution paths yet share similar average performance levels.
Additionally, during the saturation stage, all potential merges display a strong affinity, with model kinship values nearing 1.
Since model kinship indicates the similarity of weights, we concluide the final findings as:

## 4 Using Model Kinship to Improve Model Merging

Inspired by the above findings, we further leverage model kinship to enhance the model merging process.
We firstly conduct experiments employing a performance-prior greedy merging strategy.
Note that the greedy strategy may eventually lead to convergence, we further introduce Top-$k$ Greedy Merging with Model Kinship (Algorithm 1).
Our results indicate that while the greedy strategy focuses on short-term gains, it can lead to parameter convergence and suboptimal outcomes.
By integrating model kinship, we can help the strategy avoid local optima.
Furthermore, we find that model kinship holds potential for enhancing merging strategies as an early stopping criterion.

### 4.1 Experiment Setup

Figure: Algorithm 1 Top $k$ Greedy Merging with Model Kinship.

Figure: Figure 6: Left (a): The comparison of task performance improvement across merging generations. The red curve (greedy strategy) saturates by generation 2, while the blue curve (modified strategy) escapes the local optima at generation 2 and continues improving multitask capabilities. Right (b): The partial model family tree from the controled experiments. The red arrow shows the critical change between experiment 1 and experiment 2 in the evolution path.
Refer to caption: x5.png

##### LLMs.

We select three fine-tuned, open-source LLMs based on the Mistral-7B architecture from HuggingFace: mistral-7b-instruct-v0.2, metamath-mistral-7b, and open-chat-3.5-1210.
Each model demonstrates distinct strengths across three benchmarks: Winogrande (5-shot), GSM8k (5-shot), and TruthfulQA MC2 (0-shot).
All models are evaluated using the Language Model Evaluation Harness (Gao et al., ), a widely adopted framework for testing LLMs.

##### Merging Method.

We conduct two iterative model merging experiments, both utilizing the SLERP (Spherical Linear Interpolation) (Shoemake, ) for merging(^3^33We provide a Google Colab [Notebook](https://colab.research.google.com/drive/141VAI89emgSIcwkswATEXSEENoAMywTO?usp=sharing).).
For implementation, we employ Mergekit (Goddard et al., ), a comprehensive toolkit that offers simple access to state-of-the-art model merging techniques.

##### Top k 𝑘 k italic_k Greedy Merging.

This strategy utilizes the vanilla Top-$k$ Greedy Merging approach on $n$ LLMs (as outlined in the black section of Algorithm 1).
This approach has been widely adopted in the community and has demonstrated notable success.
In Figure 6 (b), models generated by the greedy strategy are indicated in green, while the best-performing models are highlighted in red.

##### Top k 𝑘 k italic_k Greedy Merging with Model Kinship.

The propposed strategy simply introduces an additional exploration step, based on model kinship, to the original greedy strategy (highlighted by the blue part in Algorithm 1).
This approach aims to merge the best-performing model with the model that has the most distinct task capabilities, in order to discover potentially better solutions.
In Figure 6 (b), models generated by our strategy are marked in purple, while the best-performing models are marked in yellow.

### 4.2 Results and Discussion

Figure 6 (a) illustrates the improvements in top average task performance across merging generations.
Table 2 provides the model average task performance, merge gain, and model kinship for each generation, comparing the original greedy merging strategy with our kinship-based method.

Both strategies achieve the multitask goals. However, the vanilla greedy strategy stops improving after Generation 2, stabilizing at an average task performance of 68.72. In contrast, Experiment 2, utilizing model kinship-based exploration, escapes the local optima (Model-2-1) and continues to improve, reaching 69.13 by Generation 5.

**Table 2: Results of merging experiments comparing the vanilla greedy strategy and our proposed approach. The first three models serve as the foundation models in both experiments. Merged models are labeled using the structure: Model-{generation number}-{model identification number}.**
| Greedy Strategy | + Model Kinship |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Model | Avg. | Gain | Kinship | Model | Avg. | Gain | Kinship |
| MetaMath | 63.72 | / | / | MetaMath | 63.72 | / | / |
| Instruct | 61.82 | / | / | Instruct | 61.82 | / | / |
| Open-chat | 66.28 | / | / | Open-chat | 66.28 | / | / |
| model-1-1 | 62.17 | -0.6 | 0.01 | model-1-1 | 62.17 | -0.6 | 0.01 |
| model-1-2 | 64.02 | -0.03 | -0.02 | model-1-2 | 64.02 | -0.03 | -0.02 |
| model-1-3 | 66.84 | +1.84 | 0.05 | model-1-3 | 66.84 | +1.84 | 0.05 |
| model-2-1 | 68.72 | +2.16 | 0.93 | model-2-1 | 68.72 | +2.16 | 0.93 |
| model-2-2 | 61.47 | -3.96 | 0.57 | model-2-2 | 61.47 | -3.96 | 0.57 |
| model-2-3 | 61.32 | -3.83 | 0.58 | model-2-3 | 61.32 | -3.83 | 0.58 |
| model-3-1 | 68.59 | +1.09 | 0.95 | model-3-2 | 67.74 | +1.09 | 0.93 |
| model-3-2 | 67.74 | -0.04 | 0.93 | model-3-3 | 69.06 | +0.74 | 0.24 |
|  | - | - | - | model-3-4 | 68.61 | +1.13 | 0.32 |
| model-4-1 | 68.51 | -0.14 | 0.98 | model-4-4 | 68.75 | -0.14 | 0.54 |
| model-4-2 | 68.04 | -0.19 | 0.98 | model-4-5 | 68.39 | -0.27 | 0.66 |
| model-4-3 | 68.53 | +0.37 | 0.94 | model-4-6 | 69.03 | +0.15 | 0.52 |
|  | - | - | - | model-5-1 | 69.13 | +0.04 | 0.65 |
|  | - | - | - | model-5-2 | 68.98 | +0.07 | 0.65 |
|  | - | - | - | model-5-3 | 68.63 | -0.37 | 0.98 |

Figure: Figure 7: Weight Change.
Refer to caption: x6.png

##### Merging Models with Low Kinship can Boost Exploration.

Figure 6(b) highlights the key branch of the model family tree.
To investigate how merging models with low kinship helps escape local optima, we focus on the bifurcation point and analyze the weight changes: $v_{1}$ (from Model-2-1 to Model-3-1) and $v_{2}$ (from Model-2-1 to Model-3-3) in two separate experiments. The previous weight change, $v_{pre}$ (from Model-1-3 to Model-2-1), serves as a baseline.
Figure 7 reveals that merging with the exploration model resulted in significant weight changes in a distinct direction, introducing novel variations into the weight space.
In contrast, $v_{1}$ shows minimal weight change, as the merging effect is reduced due to the high similarity between the weights of openchat-3.5 and Model-2-1.

##### Early Stopping at High Kinship can Improve Efficiency.

We observe that the saturation stage of model evolution is particularly resource-intensive.
In community experiments, 5 out of 14 merges in evolution path 1 resulted in an average improvement of just 0.57, while 3 out of 12 merges in evolution Path 2 yields an average improvement of 0.36.
In our own experiments, applying a greedy strategy to a simple task lead to saturation after 2 out of 4 merges, with no further gains.
These results indicate that human judgment and conventional stopping conditions cannot effectively halt the merging process at the optimal time.
Therefore, we propose that model kinship can be used as an effective early stopping signal.
When merging converges, the model kinship between top-performing models often exceeds 0.9.
By halting the merging process at this point, time efficiency improves by approximately 30%, with minimal or no reduction in performance.

## 5 Related work

Weight averaging is one of the most widely used techniques in model merging, with its origins traced back to Utans (), who first applied it in neural networks to achieve performance comparable to ensemble methods.
Since the 2010s, weight averaging has found numerous applications in deep neural networks, including combining checkpoints to enhance the training process (Nagarajan & Kolter, ; Tarvainen & Valpola, ; Izmailov et al., ; Li et al., ; Stoica et al., ; Padmanabhan et al., ; Jang et al., ), leveraging task-specific information (Li et al., ; Smith & Gashler, ; Ilharco et al., ; Izmailov et al., ), and parallel training of large language models (LLMs) (Li et al., ).
Discovery of Linear Mode Connectivity (LMC) (Garipov et al., ; Frankle et al., ; Entezari et al., ) further expands the use of weight averaging in fusing fine-tuned models through averaging methods (Neyshabur et al., ; Wortsman et al., ).
Further studies have explored optimizable weights for merging, such as Fisher-Merging (Matena & Raffel, ), RegMean (Jin et al., ), AdaMerging (Yang et al., ), MaTS (Tam et al., ).
Ilharco et al. () introduce task vectors, representing the weight difference between a fine-tuned model and its base.
They demonstrate that arithmetic operations on these vectors enable model editing, such as achieving multitask learning.
Further research on parameter interference led to TIES (Yadav et al., ), which preserves important weights and reduces sign conflicts, and DARE (Yu et al., ), which prevents interference by randomly dropping weights.
The Model Breadcrumbs (Davari & Belilovsky, ) show that the removal of outliers in parameters can reduce noise in model merging.
Merging models with different initializations requires additional considerations.
Common methods exploit the permutation symmetry of neural networks (Ainsworth et al., ; Tatro et al., ; Singh & Jaggi, ; Guerrero-Peña et al., ), using alignment techniques to mitigate the interpolation barrier (Xu et al., ; Navon et al., ).
While weight averaging cannot be directly applied to models with different architectures, it can still be used to enhance feasible fusion methods.
Recent work, such as FuseChat (Wan et al., ), combines weight averaging with Knowledge Fusion (Wan et al., ) to develop innovative fusion techniques.

Recently, there have been some works exploring “model evolution”.
Tellamekala et al. () propose the CoLD Fusion method, showing that iterative fusion can create effective multitask models.
Labonne () develop a tool to automatically merge models on Hugging Face, using an “Automerge” experiment to explore metrics in the merging process.
Akiba et al. () introduce Evolutionary Model Merge, employing evolutionary techniques to optimize model combinations, arguing that human intuition alone cannot uncover hidden patterns in merging.

## 6 Conclusion and Limitations

In this paper, we introduce model kinship, the degree of similarity or relatedness between LLMs, for merging LLMs, which can help guide our selection of candidate models.
We conduct comprehensive experiments to demonstrate its effectiveness in understanding the model evolution process.
We further propose a new model merging strategy: Top-$k$ Greedy Merging with Model Kinship.
We show that model kinship plays a crucial role in model evolution by guiding the process to escape local optima traps (in saturation stage), enabling further improvements.
Additionally, we demonstrate that model kinship can detect the onset of convergence, allowing for early stopping and reducing the waste of computational resources in the merging process.

In a broad sense, our work explores how models can achieve autonomous evolution through model merging.
Model merging can, to some extent, be likened to biological hybridization.
Biological organisms have undergone billions of years of evolution to reach their current state.
However, how silicon-based intelligence, represented by LLMs, evolves remains an unresolved mystery.
We aspire that this work offer guidance and insights for the future merging and evolution of LLMs.

## Limitations

However, there are several limitations to consider:
a) The experiments in this study are conducted exclusively on models with the Mistral architecture, leaving uncertainty about the transferability of our metric and method to other architectures, such as Mamba (Gu & Dao, ).
b) The analysis relies on open-source data from the Open Leaderboard, which is community-generated and may contain noise due to user bias.
c) Correlation metrics for model kinship have not been fully explored.
Other metrics may perform better than those discussed in this paper.
d) The effectiveness of model kinship is demonstrated through empirical evidence.
However, a theoretical framework (such as the assumptions in Appendix C) is needed to explain model evolution and model kinship more rigorously.
e) Model kinship currently guides merging and improves performance limits but does not support sustained evolution. Future progress may require environmental feedback, reward models (Silver et al., ), as well as new architectures.

## Reproducibility Statement

The experimental setup can be found in Section 4.1.
All model checkpoints are available on HuggingFace, with detailed information provided in Appendices A and B.

## Appendix A Details of Experiments

All merged models from these experiments are accessible through the Hugging Face Hub(^4^44[https://huggingface.co/datasets](https://huggingface.co/datasets)). The following tables cover two primary aspects:

- •
(1) Information on the selected model family trees for two distinct evolution paths, along with detailed analysis results for each merge.
- •
(2) A summary of the merge experiments conducted for distribution analysis.

The evolution paths are selected using a structured process, focusing on identifying key sequences within the model family trees. The steps were as follows:

- •
Model Family Tree Construction: The complete model family tree is constructed by referencing model card details for each model involved.
- •
Branch Identification: We identified the two longest branches within each tree, representing significant sequences of model merging.
- •
Performance and Kinship Evaluation: These branches were analyzed for changes in merging performance, particularly focusing on shifts in multitask capabilities and model kinship metrics.

Table 3 and 4 present detailed information on the sequential merging process. The second and third columns record the foundational models involved in each merge, while the final column lists the resulting merged models.

**Table 3: Model Family tree of evolution Path 1.**
| Gen | Model-1 | Model-2 | Model-Merged |
| --- | --- | --- | --- |
| 1 | Marcoroni-7B-v3 | Mistral-7B-Merge-14-v0.1 | distilabeled-Marcoro14-7B-slerp |
| 2 | distilabeled-Marcoro14-7B | UNA-TheBeagle-7b-v1 | Beagle14-7B |
| 3 | NeuralBeagle14-7B | Turdus | TurdusBeagle-7B |
| 4 | TurdusBeagle-7B | FernandoGPT-v1 | StrangeMerges_9-7B-dare_ties |
| 5 | StrangeMerges_9-7B-dare_ties | MBX-7B-v3 | StrangeMerges_10-7B-slerp |
| 6 | StrangeMerges_10-7B-slerp | NeuralBeagle14-7B | StrangeMerges_11-7B-slerp |
| 7 | StrangeMerges_11-7B-slerp | MBX-7B-v3 | StrangeMerges_20-7B-slerp |
| 8 | StrangeMerges_20-7B-slerp | NeuTrixOmniBe-7B-model | StrangeMerges_21-7B-slerp |
| 9 | StrangeMerges_21-7B-slerp | Experiment26 | StrangeMerges_30-7B-slerp |
| 10 | StrangeMerges_30-7B-slerp | Experiment24 | StrangeMerges_31-7B-slerp |
| 11 | StrangeMerges_31-7B-slerp | Experiment28 | StrangeMerges_32-7B-slerp |
| 12 | StrangeMerges_32-7B-slerp | … | shadow-clown-7B-slerp |
| 13 | shadow-clown-7B-slerp | yam-jom-7B | YamShadow-7B |
| 14 | YamShadow-7B | Experiment28 | YamshadowExperiment28-7B |

**Table 4: Model Family tree of evolution Path 2.**
| Gen | Model-1 | Model-2 | Model-Merged |
| --- | --- | --- | --- |
| 1 | neural-chat-7b-v3-3 | openchat-3.5-1210 | CatPPT-base |
| 2 | Marcoroni-7B-v3 | CatPPT-base | CatMacaroni-Slerp |
| 3 | LeoScorpius-7B | CatMacaroni-Slerp | SamirGPT-v1 |
| 4 | SamirGPT-v1 | … | Daredevil-7B |
| 5 | NeuralBeagle14-7B | NeuralDaredevil-7B | DareBeagle-7B |
| 6 | Turdus | DareBeagle-7B | TurdusDareBeagle-7B |
| 7 | MarcMistral-7B | TurdusDareBeagle-7B | MarcDareBeagle-7B |
| 8 | MarcBeagle-7B | MarcDareBeagle-7B | MBX-7B |
| 9 | MBX-7B | … | pastiche-crown-clown-7b-dare |
| 10 | pastiche-crown-clown-7b-dare | … | shadow-clown-7B-slerp |
| 11 | yam-jom-7B | shadow-clown-7B-slerp | YamShadow-7B |
| 12 | Experiment28-7B | YamShadow-7B | YamshadowExperiment28-7B |

Table 5 and Table 6 present detailed analysis results that are not reported in the main paper. These include Average Task Performance (ATP), merge gains, and model kinship values, which are computed using Pearson Correlation coefficient, Cosine Similarity, and Euclidean Distance for each merge.

**Table 5: Summary of Path 1 Results.**
| Gen | Model-Merged | ATP | Gain | PCC | CS | ED |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | distilabeled-Marcoro14-7B-slerp | 73.63 | 0.55 | 0.82 | 0.76 | 5.15 |
| 2 | Beagle14-7B | 74.74 | 1.01 | 0.81 | 0.79 | 6.43 |
| 3 | StrangeMerges_9-7B-dare_ties | 75.15 | 0.45 | 0.93 | 0.89 | 4.66 |
| 4 | StrangeMerges_9-7B-dare_ties | 73.32 | -0.69 | 0.90 | 0.84 | 4.70 |
| 5 | StrangeMerges_10-7B-slerp | 74.77 | 0.59 | 0.59 | 0.59 | 9.43 |
| 6 | StrangeMerges_11-7B-slerp | 74.8 | 0.045 | 0.87 | 0.86 | 5.31 |
| 7 | StrangeMerges_20-7B-slerp | 75.52 | 0.6 | 0.84 | 0.85 | 4.82 |
| 8 | StrangeMerges_21-7B-slerp | 76.29 | 0.38 | 0.85 | 0.89 | 4.28 |
| 9 | StrangeMerges_30-7B-slerp | 76.58 | 0.065 | 0.96 | 0.94 | 2.83 |
| 10 | StrangeMerges_31-7B-slerp | 76.67 | -0.02 | 0.97 | 0.97 | 2.21 |
| 11 | StrangeMerges_32-7B-slerp | 76.68 | 0.11 | 0.99 | 0.99 | 0.62 |
| 12 | shadow-clown-7B-slerp | 76.64 | -0.02 | 0.93 | 0.94 | 2.49 |
| 13 | YamShadow-7B | 76.6 | -0.02 | 0.97 | 0.97 | 2.19 |
| 14 | YamshadowExperiment28-7B | 76.86 | 0.25 | 0.98 | 0.98 | 1.61 |

**Table 6: Summary of Path 2 Results.**
| Gen | Model-Merged | ATP | Gain | PCC | CS | ED |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | CatPPT-base | 72.25 | 2.89 | 0.02 | 0.01 | 20.41 |
| 2 | CatMacaroni-Slerp | 72.74 | 0.35 | 0.88 | 0.83 | 6.16 |
| 3 | SamirGPT-v1 | 73.11 | 0.64 | 0.79 | 0.70 | 6.47 |
| 4 | Daredevil-7B | 74.12 | 0.33 | 0.87 | 0.83 | 4.81 |
| 5 | DareBeagle-7B | 74.58 | 0.15 | 0.79 | 0.77 | 6.01 |
| 6 | TurdusDareBeagle-7B | 74.94 | 0.32 | 0.90 | 0.86 | 4.59 |
| 7 | MarcDareBeagle-7B | 74.75 | 0.67 | 0.87 | 0.87 | 4.17 |
| 8 | MBX-7B | 75.04 | 0.11 | 0.96 | 0.96 | 2.90 |
| 9 | pastiche-crown-clown-7b-dare | 76.50 | 0.29 | 0.83 | 0.84 | 5.38 |
| 10 | shadow-clown-7B-slerp | 76.64 | -0.02 | 0.93 | 0.94 | 2.49 |
| 11 | YamShadow-7B | 76.60 | -0.02 | 0.97 | 0.97 | 2.19 |
| 12 | YamshadowExperiment28-7B | 76.86 | 0.25 | 0.98 | 0.98 | 1.61 |

Table 7 presents all merge experiments contributing to the distribution analysis. The selection of sample experiments adheres to two rules: (1) Samples are evenly chosen across average task performance values ranging from 0.7 to 0.7686 (the average task performance of the best 7B merged model) to accurately reflect the full scope of model evolution. (2) The experiments involve merges of two foundation models, as including multiple models introduces excessive noise.

**Table 7: All Sample Experiments used in distribution analysis.**
| Model 1 | Model 2 | Merge Gain |
| --- | --- | --- |
| Multi_verse_model-7B | Experiment26-7B | 0.06 |
| M7-7b | StrangeMerges_32-7B-slerp | -0.03 |
| Ognoexperiment27 | Multi_verse_model-7B | 0.03 |
| YamShadow-7B | Experiment28 | 0.25 |
| shadow-clown-7B-slerp | yam-jom-7B | -0.02 |
| StrangeMerges_21-7B-slerp | Experiment26 | 0.06 |
| StrangeMerges_31-7B-slerp | Experiment28 | 0.11 |
| NeuralBeagle14-7B | Turdus | 0.45 |
| DareBeagle-7B | Turdus | 0.32 |
| TurdusBeagle-7B | FernandoGPT-v1 | -0.69 |
| StrangeMerges_10-7B-slerp | NeuralBeagle14-7B | 0.04 |
| TurdusDareBeagle-7B | MarcMistral-7B | 0.67 |
| StrangeMerges_20-7B-slerp | NeuTrixOmniBe-7B-model-remix | 0.38 |
| StrangeMerges_11-7B-slerp | MBX-7B-v3 | 0.6 |
| Marcoroni-7B-v3 | Mistral-7B-Merge-14-v0.1 | 0.55 |
| distilabeled-Marcoro14-7B-slerp | UNA-TheBeagle-7b-v1 | 1.01 |
| UNA-TheBeagle-7b-v1 | distilabeled-Marcoro14-7B-slerp | 0.89 |
| CatPPT-base | Marcoroni-7B-v3 | 0.35 |
| CatMacaroni-Slerp | LeoScorpius-7B | 0.64 |
| NeuralDaredevil-7B | NeuralBeagle14-7B | 0.15 |
| StrangeMerges_9-7B-dare_ties | MBX-7B-v3 | 0.59 |
| mistral-ft-optimized-1218 | NeuralHerems-Mistral-2.5-7B | -0.85 |
| neural-chat-7b-v3-2 | OpenHermes-2.5-Mistral-7B | 1.91 |
| StrangeMerges_30-7B-slerp | Experiment24 | -0.02 |
| openchat-3.5-1210 | neural-chat-7b-v3-3 | 2.89 |
| MultiverseEx26-7B-slerp | CalmExperiment-7B-slerp | -0.09 |
| CapybaraMarcoroni-7B | DistilHermes-2.5-Mistral-7B | 0.47 |
| Multi_verse_model-7B | Calme-7B-Instruct-v0.9 | 0.04 |
| StrangeMerges_16-7B-slerp | coven_7b_128k_orpo_alpha | -0.35 |
| Kunoichi-DPO-v2-7B | AlphaMonarch-7B | -1.05 |
| StrangeMerges_15-7B-slerp | Kunoichi-7B | 0.39 |
| Mistral-T5-7B-v1 | Marcoroni-neural-chat-7B-v2 | -0.18 |
| Marcoro14-7B-slerp | mistral-ft-optimized-1218 | -0.61 |
| mistral-ft-optimized-1218 | NeuralHermes-2.5-Mistral-7B | -0.85 |
| MarcDareBeagle-7B | MarcBeagle-7B | -0.07 |
| MetaMath-Mistral-7B | Tulpar-7b-v2 | -0.29 |
| YugoGPT | AlphaMonarch-7B | -5.96 |

## Appendix B Details of Model Group Selection

This appendix presents the exact models included in each model group, as shown in Table 8. The selection process is conducted across three distinct groups: (1) the top 5 models on the leaderboard, with a performance difference of 0.2, (2) 5 models with performance scores around 73 and a performance difference of 0.2, and (3) 5 fine-tuned models. It is important to note that the fine-tuned models were not selected based on performance, and may exhibit significant differences in results.

**Table 8: Model Group in Kinship Matrix Analysis.**
| Group | Models |
| --- | --- |
| Top Model Group | YamshadowExperiment28-7B |
| Yamshadow-7B |  |
| Experiment25-7B |  |
| StrangeMerges_24-7B-slerp |  |
| MonaTrix-v6 |  |
| Mid Stage Model Group | Daredevil-7B |
| CatMarcoro14-7B |  |
| Mayo |  |
| Calmesmol-7B-slerp |  |
| StrangeMerges_4-7B-slerp |  |
| Fine-tuned Model Group | Zephyr-beta |
| MetaMath-Mistral-7B |  |
| Mistral-7B-Instruct-v0.2 |  |
| openchat-3.5-1210 |  |
| WizardLM-2 |  |

## Appendix C Assumption of Continual Model Merging

Figure: Figure 8: An intuitive illustration of the optimization process assumption in model evolution, where models progressively converge towards the optimal model.
Refer to caption: extracted/5926041/Figures/Intutive.png

Our findings in the main paper offer a new perspective on model evolution through multiple merging. If the merging process can be improved using a common optimization strategy, it raises the question of whether the underlying mechanism mirrors this optimization problem.
Thus, we hypothesize the following:

Figure: Figure 9: An intuitive illustration of how model evolution can fall into local optima due to a performance-prior strategy. It shows that Merged Model 2 may be overlooked, despite its potential for better multitask performance.
Refer to caption: extracted/5926041/Figures/additional_intitutive.png

Figure 8 illustrates the ideal scenario in our assumption where multiple merges produce a highly generalized model. For the generalization task $t$, the y-axis represents the model performance for task $t$ and the x-axis represents the model’s weight space. In early merging stages, models fine-tuned with different tasks exhibit significant weight space dissimilarity. The merging process averages these weight spaces, and the experiment conductor selects the better-merged models while discarding the inferior ones. In stage 2, the search area narrows and the improvements become stable, eventually leading to an optimized state in stage 3 when “saturation stage” occurs.

In this context, Model Kinship serves as a metric to quantify the weight space distance between two models, with a higher model kinship indicating a lower weight space distance. Following this assumption, our findings of the optimization problem in model evolution can be elucidated in Figure 9.

However, we currently lack sufficient evidence to validate this hypothesis.
Future work is needed to explore this further.

## Appendix D Other Information and Results

This section presents additional results that are not reported in the main paper but may be useful for those interested. Figure 10 compares average task performance and model kinship using alternative metrics. Table 9 provides information on the exploration models used in our experiments. Figure 11 depicts a more “realistic” representation of combining two features/capabilities in evolution, highlighting the challenges in the selection process.

Figure: Figure 10: Illustration of comparison between the correlation of Pearson Correlation Coefficient (PCC), Cosine Similarity (CS), and Euclidean Distance (ED) with average task performance (Normalized to the same value scale).
Refer to caption: x7.png

**Table 9: Results of Exploration Models in our modified strategy.**
| Model | Avg. | Gain | Kinship |
| --- | --- | --- | --- |
| model-2-exp | 68.11 | +3.78 | -0.02 |
| model-3-exp | 68.15 | +2.70 | 0.03 |
| model-4-exp | 68.49 | +3.42 | 0.48 |
| model-5-exp | 68.36 | -0.09 | 0.53 |

Figure: Figure 11: An intuitive comparison between selective breeding and continual model merging. The left process demonstrates breeding a tall and frutful plant by selecting parents with the desired traits in an biological scenario. The right process shows developing a model with capabilities of coding and math through model evolution.
Refer to caption: extracted/5926041/Figures/Breeding.png

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 3
presentation: 2
questions: As mentioned in the above weaknesses.
soundness: 3
strengths: 1. The research area is significant, and the introduction of the kinship metric, drawing an analogy to biological evolution, presents a human-like perspective that adds depth to the discussion of model merging.

2. The paper effectively establishes a connection between kinship and performance gains, providing empirical evidence that supports the proposed approach.
summary: This paper introduces the concept of model kinship, drawing an analogy to genetic hybridization in biology, with the aim of providing guidance for effective performance improvements in the field of model merging. The authors first empirically validate the relationship between kinship and performance gains, demonstrating their consistency. Building on the concept of kinship, they propose a Top-k Greedy Merging method that shows some performance enhancements compared to baseline approaches.
weaknesses: 1. The definition of the problem is unclear, and the paper lacks a thorough discussion of specific scientific issues within the model merging field. The research motivation is insufficient, leading to a lack of logical clarity in the arguments presented.

2. While introducing human-like concepts or analogies from other fields can be a valuable approach to problem-solving, the kinship metric designed in this paper lacks a reasonable formulation and effective validation, which undermines its applicability.

3. The logical coherence and rigor of the writing require significant improvement to enhance the overall quality of the paper.

4. The conclusions drawn in the paper lack persuasive power due to the absence of quantitative experimental analysis. The methodology and experimental design appear relatively crude, which detracts from the credibility of the findings.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: Please see above weakness.
soundness: 3
strengths: This paper targets at solving an important task. Also, this paper is very complete and well-written, from introducing the concept of model kinship, to empirically verify its value, and finally provide a practical scenario to apply model kinship to improve the model merging performance. I feel this is a complete work.
summary: This paper proposes model kinship to evaluate the degree of similarity between LLMs. Then the authors did experiments to validate that model kinship moderately correlates with merge gain, and found that the performance improvement during model merging can be divided into learning stage and saturation stage. Finally, the paper proposes a practical use case to apply model kinship into top-k greedy merging to improve the performance of model evolution.
weaknesses: 1. The bold fonts in this paper emphasize the relationship between model merging and biological evolution. Actually, I don't quite get their correlation. Biological evolution is based on nature selection while in this paper, model merging is done based on model similarity. Can the authors further elaborate on this?

2. Since all the experiments in this paper are only verified on one model set and one task set, there is no guarantee that the proposed pipeline can be generalized to other real-world scenarios.

---

confidence: 2
contribution: 3
initial_rating: 5
presentation: 2
questions: 1. Please respond to the weaknesses mentioned in my review. I will adjust the score accordingly based on your response.
soundness: 3
strengths: 1. The paper introduces a novel exploration of the kinship between LLMs, using a heuristic approach inspired by biological evolution to guide the selection of models for merging. The overall idea is interesting.

2. The narrative of the paper is well-structured, and the authors validate their experiments using Mistral as the primary architecture.
summary: 1. The paper explores the concept of "model kinship," which measures the similarity or relatedness between large language models (LLMs), drawing a parallel to biological evolution. Through empirical analysis, the authors demonstrate that this kinship can predict the performance gains when merging models, offering a basis for selecting suitable candidates for merging.

2. Building on this, they propose a strategy called Top-k Greedy Merging with Model Kinship, which improves performance on benchmark datasets. The study shows that using model kinship not only helps avoid local optima during model evolution but also serves as an early stopping criterion, enhancing efficiency. The authors suggest that their findings provide valuable insights for future LLM merging and evolution.
weaknesses: 1.  The experimental section lacks clarity. It is recommended that the authors add a dedicated subsection analyzing the datasets used and explaining why these specific datasets (e.g., general-purpose or domain-specific test data) were chosen for performance evaluation.

2. In the appendix (Figure 8), the authors present performance analysis based on various merging experiments using the Mistral architecture. However, it is unclear whether the results are applicable to other architectures like LLaMA, which suggests the idea may have some limitations. Although the authors mention in the limitations section that the performance on other architectures remains unknown, I strongly recommend testing on LLaMA or other types of LLM architectures to make the argument more robust and solid.

3. In Algorithm 1, the authors use Greedy Merging with Model Kinship; it is suggested to include an ablation study to demonstrate that the greedy algorithm is indeed the optimal choice.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: 086-087 should have a comma after improvements

table 1 and the significance of the p values for absolute gain seem to indicate that the variability of the merge gain is inversely related to the degree of model kinship. it may be worth remarking on this around line 233. 

297-299 make references to a “notable correlation” between model kinship and average task performance in figure 4, but no formal correlation analysis is presented.

311 makes reference to 3 learning stages, but line 086 makes reference to only two. this complicates the narrative

345 comma after convergence should be a period.

algorithm 1 has what seems to be some typos:

- should line 2 have M_k, not M_n, as the last element in the set?

- should line 3 replace “top m” with “top k”?

- should line 7 replace “select k pairs” with “select k models?”

- should line 9 be “identify the model M_f \in S with the lowest kinship…”? the discussion on lines 414-416 seems to indicate so.

- line 11 should expand on what “model-gen-id” is

line 429 seems to have an unlinked figure reference (perhaps to figure 6?)
soundness: 3
strengths: The paper posits and provides evidence for the importance of kinship aware strategies when performing iterative model merging. This is a highly active and impactful area of research, and the paper presents empirical results with sufficient theoretical justification that they should be noted by the broader research community.
summary: This paper introduces a notion of model kinship, which attempts to measure the relatedness of two language models. The paper also presents an algorithm that leverages the degree of model kinship to inform model merging strategies. In particular, the paper suggests that merging highly performant models with low kinship to a target model outperforms greedily merging with the top performing model when iteratively merging models. The results in the paper suggest that a model kinship aware strategy for model merging leads to superior benchmark results over model kinship unaware strategies.
weaknesses: The paper lacks experimental breadth, and in some cases, seems to have critical typos. The paper would benefit from expanded experiments, particularly with additional merge lineage studies and with simulations on base models beyond Mistral 7b.

Meta-review: The paper introduces a notion of model kinship, inspired by concepts in biological evolution, measuring how two language models are related. The authors draw a connection between this metric and the gain from merging model, and propose an algorithm exploiting this connection. The reviews agree that the solved problem is important and appreciate the novelty of the proposed metric. The main concern raised in the reviews relates to the experiments. The reviewers mentioned (1) a lack of clarity in the experiment presentation, (2) that the existing experiments are too limited in scope, and (3) the experiments should better prove the claims made in the paper. 

The authors provided a thorough rebuttal, clarifying the vague issues in the existing experiments and adding new experiments giving a better proof of generalizability of their claims. While some of the reviewers were convinced by this response and raised their score, others are still concerned that the empirical section requires more work. 

Given the remaining mixed reviews, and the large scope of the changes required, I think its best to be cautious and have the paper go through a new round of reviews. The paper has potential, especially with the added content, but the change it requires seems to large to be done without being properly reviewed.

Decision: Reject