|
--- |
|
license: cc-by-nc-4.0 |
|
tags: |
|
- merge |
|
- conversational |
|
- multi-task |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
# Winter Garden 7B - Γ |
|
|
|
It was mentioned that we are in the open ai dark winter; so I thought I would make myself a nice winter garden. |
|
|
|
## An experiment |
|
|
|
This time I did something a bit different. |
|
|
|
* Mistral-7B-v0.1 |
|
|
|
and merged in |
|
|
|
* Yarn-Mistral-7b-128k |
|
* Thespis-Balanced-7b-v1 |
|
* ZySec-7B-v1 |
|
* LemonadeRP-4.5.3 |
|
* Noromaid-7B-0.4-DPO |
|
* Prima-LelantaclesV6-7b |
|
* West-Hermes-7B |
|
* Capricorn-7B-DPO |
|
* kun-kunoichi-v1-DPO-v2-SLERP-7B |
|
* Kunoichi-DPO-v2-7B |
|
* WestLake-7B-v2-laser-truthy-dpo |
|
* StrangeMerges_6-7B-dare_ties |
|
* NeuralMarcoro14-7B |
|
* multi_verse_model |
|
* Multi-Verse-RP-7B |
|
* MonarchLake-7B |
|
* AlphaMonarch-7B |
|
|
|
in a consecutive dare ties merge, ordering the merge order by tensor-relative cosine similarity. |
|
|
|
## Chat Template |
|
|
|
Basic Mistral `<s>[INST][/INST]` works pretty well. It seems smart, but we will see. |
|
|
|
## Scores |
|
|
|
Metric | Score |
|
---|--- |
|
|
|
Average | |
|
ARC | |
|
HellaSwag | |
|
MMLU | |
|
TruthfulQA | |
|
Winogrande | |
|
GSM8K | |
