README.md

---
base_model:
- yuvraj17/Llama-3-8B-spectrum-25
- ruggsea/Llama3-stanford-encyclopedia-philosophy-QA
- arcee-ai/Llama-3.1-SuperNova-Lite
tags:
- merge
- mergekit
- lazymergekit
- yuvraj17/Llama-3-8B-spectrum-25
- ruggsea/Llama3-stanford-encyclopedia-philosophy-QA
- arcee-ai/Llama-3.1-SuperNova-Lite
license: apache-2.0
language:
- en
pipeline_tag: text-generation
---

# Llama3-8B-SuperNova-Spectrum-dare_ties

Llama3-8B-SuperNova-Spectrum-dare_ties is a `dare_ties` merge of the following models using [LazyMergekit](https://colab.research.google.com/drive/1obulZ1ROXHjYLn6PPZJwRR6GzgQogxxb?usp=sharing):
* [yuvraj17/Llama-3-8B-spectrum-25](https://huggingface.co/yuvraj17/Llama-3-8B-spectrum-25)
* [ruggsea/Llama3-stanford-encyclopedia-philosophy-QA](https://huggingface.co/ruggsea/Llama3-stanford-encyclopedia-philosophy-QA)
* [arcee-ai/Llama-3.1-SuperNova-Lite](https://huggingface.co/arcee-ai/Llama-3.1-SuperNova-Lite)

## DARE_TIES Merging

### TIES Merging

[TIES](https://arxiv.org/abs/2306.01708) Merging, introduced by Yadav et al. (2023), is a method for merging multiple specialized models into one general-purpose model. It solves two key challenges:
* **Redundancy Removal**: Identifies and eliminates overlapping or unnecessary information between models, making the final model more efficient.
* **Conflict Resolution**: Reconciles differences between models by creating a unified sign vector that represents the most dominant direction of change across all models.

**TRIES** stands for **T**R**I**M, **E**LECT **S**IGN & MERGE (TIES-MERGING). 

<figure>

  <img src="https://cdn-uploads.huggingface.co/production/uploads/66137d95e8d2cda230ddcea6/2vBgcGko-tcsaAkLUzHnU.png" width="1000" height="768">
  <figcaption> How TIES-Merging Works <a href="//arxiv.org/pdf/2306.01708">Reference</a> </figcaption>

</figure>


### DARE Merging

Introduced by Yu et al. (2023), [DARE](https://arxiv.org/abs/2311.03099) uses an approach similar to TIES with two main differences:

* **Weight Pruning**: Randomly resets some fine-tuned weights to their original values, reducing model complexity.
* **Weight Scaling**: Adjusts the remaining weights by scaling and combining them with the base model's weights to maintain consistent performance.

**DARE** stands for **D**ROP **A**ND **RE**SCALE

Mergekit’s implementation of DARE-Merging has two flavours: with the sign election step of TIES (`dare_ties`) or without (`dare_linear`). I have chosen `dare_ties` for this merge.

For more information refer this [Merge Large Language Models with MergeKit by Maxime Labonne](https://towardsdatascience.com/merge-large-language-models-with-mergekit-2118fb392b54)
 
Also, if you want to get in-depth knowledge about Model-Merging and its different types, I highly recommend this [YouTube Video by Julien Simon](https://youtu.be/cvOpX75Kz4M?si=d5crVWSxcjvNUm6a)

## 🧩 Configuration

```yaml
models:
  - model: NousResearch/Meta-Llama-3-8B
    # No parameters necessary for base model
  - model: yuvraj17/Llama-3-8B-spectrum-25
    parameters:
      density: 0.56
      weight: 0.12
  - model: ruggsea/Llama3-stanford-encyclopedia-philosophy-QA
    parameters:
      density: 0.56
      weight: 0.12
  - model: arcee-ai/Llama-3.1-SuperNova-Lite
    parameters:
      density: 0.58
      weight: 0.55
merge_method: dare_ties
base_model: NousResearch/Meta-Llama-3-8B
dtype: bfloat16
```

## 💻 Usage

```python
!pip install -qU transformers accelerate

from transformers import AutoTokenizer
import transformers
import torch

model = "yuvraj17/Llama3-8B-SuperNova-Spectrum-dare_ties"
messages = [{"role": "user", "content": "What is a large language model?"}]

tokenizer = AutoTokenizer.from_pretrained(model)
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
pipeline = transformers.pipeline(
    "text-generation",
    model=model,
    torch_dtype=torch.float16,
    device_map="auto",
)

outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
```

## 🏆 Evaluation Scores

### Nous

|                                                     Model                                                      |AGIEval|TruthfulQA|Bigbench|
|----------------------------------------------------------------------------------------------------------------|------:|---------:|-------:|
|[Llama3-8B-SuperNova-Spectrum-dare_ties](https://huggingface.co/yuvraj17/Llama3-8B-SuperNova-Spectrum-dare_ties)|  38.32|     57.15|   43.91|

### AGIEval
|             Task             |Version| Metric |Value|   |Stderr|
|------------------------------|------:|--------|----:|---|-----:|
|agieval_aqua_rat              |      0|acc     |20.47|±  |  2.54|
|                              |       |acc_norm|18.50|±  |  2.44|
|agieval_logiqa_en             |      0|acc     |35.94|±  |  1.88|
|                              |       |acc_norm|35.64|±  |  1.88|
|agieval_lsat_ar               |      0|acc     |21.74|±  |  2.73|
|                              |       |acc_norm|20.00|±  |  2.64|
|agieval_lsat_lr               |      0|acc     |41.37|±  |  2.18|
|                              |       |acc_norm|40.98|±  |  2.18|
|agieval_lsat_rc               |      0|acc     |59.11|±  |  3.00|
|                              |       |acc_norm|56.13|±  |  3.03|
|agieval_sat_en                |      0|acc     |63.59|±  |  3.36|
|                              |       |acc_norm|60.19|±  |  3.42|
|agieval_sat_en_without_passage|      0|acc     |40.29|±  |  3.43|
|                              |       |acc_norm|37.38|±  |  3.38|
|agieval_sat_math              |      0|acc     |38.64|±  |  3.29|
|                              |       |acc_norm|37.73|±  |  3.28|

Average: 38.32%

### TruthfulQA
|    Task     |Version|Metric|Value|   |Stderr|
|-------------|------:|------|----:|---|-----:|
|truthfulqa_mc|      1|mc1   |38.43|±  |   1.7|
|             |       |mc2   |57.15|±  |   1.5|

Average: 57.15%

### Bigbench
|                      Task                      |Version|       Metric        |Value|   |Stderr|
|------------------------------------------------|------:|---------------------|----:|---|-----:|
|bigbench_causal_judgement                       |      0|multiple_choice_grade|58.42|±  |  3.59|
|bigbench_date_understanding                     |      0|multiple_choice_grade|70.73|±  |  2.37|
|bigbench_disambiguation_qa                      |      0|multiple_choice_grade|30.23|±  |  2.86|
|bigbench_geometric_shapes                       |      0|multiple_choice_grade|47.35|±  |  2.64|
|                                                |       |exact_str_match      | 0.00|±  |  0.00|
|bigbench_logical_deduction_five_objects         |      0|multiple_choice_grade|29.00|±  |  2.03|
|bigbench_logical_deduction_seven_objects        |      0|multiple_choice_grade|21.00|±  |  1.54|
|bigbench_logical_deduction_three_objects        |      0|multiple_choice_grade|51.33|±  |  2.89|
|bigbench_movie_recommendation                   |      0|multiple_choice_grade|33.20|±  |  2.11|
|bigbench_navigate                               |      0|multiple_choice_grade|55.40|±  |  1.57|
|bigbench_reasoning_about_colored_objects        |      0|multiple_choice_grade|66.35|±  |  1.06|
|bigbench_ruin_names                             |      0|multiple_choice_grade|45.76|±  |  2.36|
|bigbench_salient_translation_error_detection    |      0|multiple_choice_grade|28.26|±  |  1.43|
|bigbench_snarks                                 |      0|multiple_choice_grade|62.43|±  |  3.61|
|bigbench_sports_understanding                   |      0|multiple_choice_grade|50.30|±  |  1.59|
|bigbench_temporal_sequences                     |      0|multiple_choice_grade|48.00|±  |  1.58|
|bigbench_tracking_shuffled_objects_five_objects |      0|multiple_choice_grade|23.60|±  |  1.20|
|bigbench_tracking_shuffled_objects_seven_objects|      0|multiple_choice_grade|17.66|±  |  0.91|
|bigbench_tracking_shuffled_objects_three_objects|      0|multiple_choice_grade|51.33|±  |  2.89|

Average: 43.91%



## Special thanks & Reference
- Maxime Labonne for their easy-to-use colab-notebook [Merging LLMs with MergeKit](https://github.com/mlabonne/llm-course/blob/main/Mergekit.ipynb), [Blog](https://towardsdatascience.com/merge-large-language-models-with-mergekit-2118fb392b54) and [LLM-AutoEva Notebookl](https://github.com/mlabonne/llm-autoeval)
- Authors of [Mergekit](https://github.com/arcee-ai/mergekit)