File size: 1,105 Bytes
ceab948 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 |
---
license: llama2
---
# LM-cocktail 10.7B v1
This is a 50%-50% model of the SOLAR model and meow.
https://huggingface.co/upstage/SOLAR-10.7B-Instruct-v1.0
https://huggingface.co/rishiraj/meow
who rank #1 and #2 among models <13B in the https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard by 2023/12/20.
# Code
The LM-cocktail is novel technique for merging multiple models https://arxiv.org/abs/2311.13534
Code is backed up by this repo https://github.com/FlagOpen/FlagEmbedding.git
Merging scripts available under the [./scripts](./scripts) folder
# Result
The SOLAR model is the first model <30B that can answer this question from my test:
```
What will AI be like in the year 1010 A.D?
```
without hullicinating into 1010 A.D is a future time (like other llama2 models)
Models greater than that, like Yi-34B could answer this paradoxic question correctly as well, since it is huge enough.
### SOLAR 10.7B output
### LMCocktail 10.7B output1
### LMCocktail 10.7B output2
 |