ChuckMcSneed
commited on
Commit
•
17d638c
1
Parent(s):
781a558
Update README.md
Browse files
README.md
CHANGED
@@ -6,6 +6,8 @@ tags:
|
|
6 |
---
|
7 |
# BETTER THAN GOLIATH?!
|
8 |
I've merged [Euryale-lora that I made](https://huggingface.co/ChuckMcSneed/Euryale-1.3-L2-70B-LORA) with [Xwin](https://huggingface.co/Xwin-LM/Xwin-LM-70B-V0.1) and then merged it with itself in [goliath-style merge](/config.yml) using [mergekit](https://github.com/arcee-ai/mergekit). The resulting model performs better than [goliath](https://huggingface.co/alpindale/goliath-120b) on my tests(note: performance on tests is not necessarily performance in practice). Test it, have fun with it. This is a sister model of [Premerge-EX-EX-123B](https://huggingface.co/ChuckMcSneed/Premerge-EX-EX-123B).
|
|
|
|
|
9 |
# Ideas behind it
|
10 |
Since the creation of Goliath I was wondering if it was possible to make something even better. I've tried linear, passthrough, SLERP, TIES-merging models, but I could not recreate the greatness of goliath, at least not in a way that I liked in practical use. I knew about the existence of LORAs but I didn't know how well they performed. I created a model named [Gembo](https://huggingface.co/ChuckMcSneed/Gembo-v1-70b) by merging a shitton of LORAs together, and surprisingly it worked! In fact it worked so well that it was the best model on my benchmarks until now. When I found a tool named [LORD](https://github.com/thomasgauthier/LoRD), which can extract LORA from any model, I knew I could do something even better.
|
11 |
|
|
|
6 |
---
|
7 |
# BETTER THAN GOLIATH?!
|
8 |
I've merged [Euryale-lora that I made](https://huggingface.co/ChuckMcSneed/Euryale-1.3-L2-70B-LORA) with [Xwin](https://huggingface.co/Xwin-LM/Xwin-LM-70B-V0.1) and then merged it with itself in [goliath-style merge](/config.yml) using [mergekit](https://github.com/arcee-ai/mergekit). The resulting model performs better than [goliath](https://huggingface.co/alpindale/goliath-120b) on my tests(note: performance on tests is not necessarily performance in practice). Test it, have fun with it. This is a sister model of [Premerge-EX-EX-123B](https://huggingface.co/ChuckMcSneed/Premerge-EX-EX-123B).
|
9 |
+
# Prompt format
|
10 |
+
Alpaca.
|
11 |
# Ideas behind it
|
12 |
Since the creation of Goliath I was wondering if it was possible to make something even better. I've tried linear, passthrough, SLERP, TIES-merging models, but I could not recreate the greatness of goliath, at least not in a way that I liked in practical use. I knew about the existence of LORAs but I didn't know how well they performed. I created a model named [Gembo](https://huggingface.co/ChuckMcSneed/Gembo-v1-70b) by merging a shitton of LORAs together, and surprisingly it worked! In fact it worked so well that it was the best model on my benchmarks until now. When I found a tool named [LORD](https://github.com/thomasgauthier/LoRD), which can extract LORA from any model, I knew I could do something even better.
|
13 |
|