Exploring Model Kinship for Merging Large Language Models

Authors: Yedi Hu, Yunzhi Yao, Ningyu Zhang, Shumin Deng, Huajun Chen
Primary area: foundation or frontier models, including LLMs
Keywords: model merging, model kinship, large language models

Abstract:
Model merging has become one of the key technologies for enhancing the capabilities and efficiency of Large Language Models (LLMs). However, our understanding of the expected performance gains and principles when merging any two models remains limited. In this work, we introduce model kinship, the degree of similarity or relatedness between LLMs, analogous to biological evolution. With comprehensive empirical analysis, we find that there is a certain relationship between model kinship and the performance gains after model merging, which can help guide our selection of candidate models. Inspired by this, we propose a new model merging strategy: Top-$k$ Greedy Merging with Model Kinship, which can yield better performance on benchmark datasets. Specifically, we discover that using model kinship as a criterion can assist us in continuously performing model merging, alleviating the degradation (local optima) in model evolution, whereas model kinship can serve as a guide to escape these traps. Furthermore, we observe that model kinship can serve as an early stopping criterion, enhancing the efficiency of  model evolution. We hope that the proposed  model kinship can provide guidance and insights for the future merging and evolution of LLMs.

Decision: reject
Reviewer scores: 3, 8, 6, 6

Reviews and discussion:
<thread>
creation date:
1729611361627

thread_type:
official_review

<thread_object>
creator: reviewer_xidk

<content>
summary:
This paper introduces a notion of model kinship, which attempts to measure the relatedness of two language models. The paper also presents an algorithm that leverages the degree of model kinship to inform model merging strategies. In particular, the paper suggests that merging highly performant models with low kinship to a target model outperforms greedily merging with the top performing model when iteratively merging models. The results in the paper suggest that a model kinship aware strategy for model merging leads to superior benchmark results over model kinship unaware strategies.

strengths:
The paper posits and provides evidence for the importance of kinship aware strategies when performing iterative model merging. This is a highly active and impactful area of research, and the paper presents empirical results with sufficient theoretical justification that they should be noted by the broader research community.

weaknesses:
The paper lacks experimental breadth, and in some cases, seems to have critical typos. The paper would benefit from expanded experiments, particularly with additional merge lineage studies and with simulations on base models beyond Mistral 7b.

questions:
086-087 should have a comma after improvements

table 1 and the significance of the p values for absolute gain seem to indicate that the variability of the merge gain is inversely related to the degree of model kinship. it may be worth remarking on this around line 233. 

297-299 make references to a “notable correlation” between model kinship and average task performance in figure 4, but no formal correlation analysis is presented.

311 makes reference to 3 learning stages, but line 086 makes reference to only two. this complicates the narrative

345 comma after convergence should be a period.

algorithm 1 has what seems to be some typos:

- should line 2 have M_k, not M_n, as the last element in the set?

- should line 3 replace “top m” with “top k”?

- should line 7 replace “select k pairs” with “select k models?”

- should line 9 be “identify the model M_f \in S with the lowest kinship…”? the discussion on lines 414-416 seems to indicate so.

- line 11 should expand on what “model-gen-id” is

line 429 seems to have an unlinked figure reference (perhaps to figure 6?)
</content>
</thread_object>

</thread>
<thread>
creation date:
1730071625572

thread_type:
official_review

<thread_object>
creator: reviewer_fhca

<content>
summary:
1. The paper explores the concept of "model kinship," which measures the similarity or relatedness between large language models (LLMs), drawing a parallel to biological evolution. Through empirical analysis, the authors demonstrate that this kinship can predict the performance gains when merging models, offering a basis for selecting suitable candidates for merging.

2. Building on this, they propose a strategy called Top-k Greedy Merging with Model Kinship, which improves performance on benchmark datasets. The study shows that using model kinship not only helps avoid local optima during model evolution but also serves as an early stopping criterion, enhancing efficiency. The authors suggest that their findings provide valuable insights for future LLM merging and evolution.

strengths:
1. The paper introduces a novel exploration of the kinship between LLMs, using a heuristic approach inspired by biological evolution to guide the selection of models for merging. The overall idea is interesting.

2. The narrative of the paper is well-structured, and the authors validate their experiments using Mistral as the primary architecture.

weaknesses:
1.  The experimental section lacks clarity. It is recommended that the authors add a dedicated subsection analyzing the datasets used and explaining why these specific datasets (e.g., general-purpose or domain-specific test data) were chosen for performance evaluation.

2. In the appendix (Figure 8), the authors present performance analysis based on various merging experiments using the Mistral architecture. However, it is unclear whether the results are applicable to other architectures like LLaMA, which suggests the idea may have some limitations. Although the authors mention in the limitations section that the performance on other architectures remains unknown, I strongly recommend testing on LLaMA or other types of LLM architectures to make the argument more robust and solid.

3. In Algorithm 1, the authors use Greedy Merging with Model Kinship; it is suggested to include an ablation study to demonstrate that the greedy algorithm is indeed the optimal choice.

questions:
1. Please respond to the weaknesses mentioned in my review. I will adjust the score accordingly based on your response.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730408814336

thread_type:
official_review

<thread_object>
creator: reviewer_vep1

<content>
summary:
This paper proposes model kinship to evaluate the degree of similarity between LLMs. Then the authors did experiments to validate that model kinship moderately correlates with merge gain, and found that the performance improvement during model merging can be divided into learning stage and saturation stage. Finally, the paper proposes a practical use case to apply model kinship into top-k greedy merging to improve the performance of model evolution.

strengths:
This paper targets at solving an important task. Also, this paper is very complete and well-written, from introducing the concept of model kinship, to empirically verify its value, and finally provide a practical scenario to apply model kinship to improve the model merging performance. I feel this is a complete work.

weaknesses:
1. The bold fonts in this paper emphasize the relationship between model merging and biological evolution. Actually, I don't quite get their correlation. Biological evolution is based on nature selection while in this paper, model merging is done based on model similarity. Can the authors further elaborate on this?

2. Since all the experiments in this paper are only verified on one model set and one task set, there is no guarantee that the proposed pipeline can be generalized to other real-world scenarios.

questions:
Please see above weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730552468478

thread_type:
official_review

<thread_object>
creator: reviewer_haon

<content>
summary:
This paper introduces the concept of model kinship, drawing an analogy to genetic hybridization in biology, with the aim of providing guidance for effective performance improvements in the field of model merging. The authors first empirically validate the relationship between kinship and performance gains, demonstrating their consistency. Building on the concept of kinship, they propose a Top-k Greedy Merging method that shows some performance enhancements compared to baseline approaches.

strengths:
1. The research area is significant, and the introduction of the kinship metric, drawing an analogy to biological evolution, presents a human-like perspective that adds depth to the discussion of model merging.

2. The paper effectively establishes a connection between kinship and performance gains, providing empirical evidence that supports the proposed approach.

weaknesses:
1. The definition of the problem is unclear, and the paper lacks a thorough discussion of specific scientific issues within the model merging field. The research motivation is insufficient, leading to a lack of logical clarity in the arguments presented.

2. While introducing human-like concepts or analogies from other fields can be a valuable approach to problem-solving, the kinship metric designed in this paper lacks a reasonable formulation and effective validation, which undermines its applicability.

3. The logical coherence and rigor of the writing require significant improvement to enhance the overall quality of the paper.

4. The conclusions drawn in the paper lack persuasive power due to the absence of quantitative experimental analysis. The methodology and experimental design appear relatively crude, which detracts from the credibility of the findings.

questions:
As mentioned in the above weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732125750572

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Global Response

comment:
We sincerely thank all reviewers for their valuable feedback on our paper.

---

# Our strength

We appreciate the reviewers for recognizing our strengths.

### **Comprehensive Work**

The paper presents a complete and well-written study, effectively introducing the concept of model kinship, empirically verifying its value, and demonstrating its practical application to enhance model merging performance. (Reviewer vep1)

### **Novel and Heuristic Approach**


The introduction of the kinship metric, inspired by biological evolution, enriches the discussion of model merging with a human-like perspective. (Reviewer haon)

The exploration of kinship between LLMs using a heuristic inspired by biological evolution is innovative. (Reviewer fHcA)

### **Impactful and Empirical Evidence**

The paper highlights the significance of kinship-aware strategies in iterative model merging, supported by empirical evidence and theoretical insights in a highly active and impactful research area. (Reviewer Xidk)

The paper establishes a clear link between model kinship and performance gains, supported by empirical evidence for the proposed approach. (Reviewer haon)

---

# Our revisions and additional experiments

We also thank the reviewers for highlighting potential issues. In response to their concerns and questions, we have revised our paper, with modifications highlighted in blue. These updates include:

- Enhance the problem definition and refine the concept formulation in the background section. (Reviewer Haon) (Page 2-3)
- Provide a detailed explanation of the datasets used in the experimental settings. (Reviewer fHcA) (Page 7)
- Improve the presentation of the algorithms employed in the main experiments. (Reviewer Xidk) (Page 8)
- Include Appendix E to explain biology-related concepts. (Reviewer Vep1) (Page 21)
- Correct the typos identified in the previous version. (Reviewer Xidk)

Notably, concerns regarding the **generalization of our method** are raised by most reviewers. To address these concerns, we present additional experiments in this section. However, due to discussion time constraints, we may not be able to cover all experiments comprehensively.

**Addtional experiments** are conducted in the following order:
- 1.Evolution experiments on the Llama-2 model
- 2.Ablation study for the greedy strategy
- 3.Evolution experiments on alternative task sets
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces a notion of model kinship, inspired by concepts in biological evolution, measuring how two language models are related. The authors draw a connection between this metric and the gain from merging model, and propose an algorithm exploiting this connection. The reviews agree that the solved problem is important and appreciate the novelty of the proposed metric. The main concern raised in the reviews relates to the experiments. The reviewers mentioned (1) a lack of clarity in the experiment presentation, (2) that the existing experiments are too limited in scope, and (3) the experiments should better prove the claims made in the paper. 

The authors provided a thorough rebuttal, clarifying the vague issues in the existing experiments and adding new experiments giving a better proof of generalizability of their claims. While some of the reviewers were convinced by this response and raised their score, others are still concerned that the empirical section requires more work. 

Given the remaining mixed reviews, and the large scope of the changes required, I think its best to be cautious and have the paper go through a new round of reviews. The paper has potential, especially with the added content, but the change it requires seems to large to be done without being properly reviewed.