Hugging Face's logo

RichardErkhov
/
Yhyu13_-_LMCocktail-10.7B-v1-gguf

GGUF
Model card Files Community
RichardErkhov commited on
Commit
bbcc6af
•
1 Parent(s): 51315ae

uploaded readme

Browse files
Files changed (1) hide show
  1. README.md +114 -0
README.md ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Quantization made by Richard Erkhov.
2
+
3
+ [Github](https://github.com/RichardErkhov)
4
+
5
+ [Discord](https://discord.gg/pvy7H8DZMG)
6
+
7
+ [Request more models](https://github.com/RichardErkhov/quant_request)
8
+
9
+
10
+ LMCocktail-10.7B-v1 - GGUF
11
+ - Model creator: https://huggingface.co/Yhyu13/
12
+ - Original model: https://huggingface.co/Yhyu13/LMCocktail-10.7B-v1/
13
+
14
+
15
+ | Name | Quant method | Size |
16
+ | ---- | ---- | ---- |
17
+ | [LMCocktail-10.7B-v1.Q2_K.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q2_K.gguf) | Q2_K | 3.73GB |
18
+ | [LMCocktail-10.7B-v1.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.IQ3_XS.gguf) | IQ3_XS | 4.14GB |
19
+ | [LMCocktail-10.7B-v1.IQ3_S.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.IQ3_S.gguf) | IQ3_S | 4.37GB |
20
+ | [LMCocktail-10.7B-v1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q3_K_S.gguf) | Q3_K_S | 4.34GB |
21
+ | [LMCocktail-10.7B-v1.IQ3_M.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.IQ3_M.gguf) | IQ3_M | 4.51GB |
22
+ | [LMCocktail-10.7B-v1.Q3_K.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q3_K.gguf) | Q3_K | 4.84GB |
23
+ | [LMCocktail-10.7B-v1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q3_K_M.gguf) | Q3_K_M | 4.84GB |
24
+ | [LMCocktail-10.7B-v1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q3_K_L.gguf) | Q3_K_L | 5.26GB |
25
+ | [LMCocktail-10.7B-v1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.IQ4_XS.gguf) | IQ4_XS | 5.43GB |
26
+ | [LMCocktail-10.7B-v1.Q4_0.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q4_0.gguf) | Q4_0 | 5.66GB |
27
+ | [LMCocktail-10.7B-v1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.IQ4_NL.gguf) | IQ4_NL | 5.72GB |
28
+ | [LMCocktail-10.7B-v1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q4_K_S.gguf) | Q4_K_S | 5.7GB |
29
+ | [LMCocktail-10.7B-v1.Q4_K.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q4_K.gguf) | Q4_K | 6.02GB |
30
+ | [LMCocktail-10.7B-v1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q4_K_M.gguf) | Q4_K_M | 6.02GB |
31
+ | [LMCocktail-10.7B-v1.Q4_1.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q4_1.gguf) | Q4_1 | 6.27GB |
32
+ | [LMCocktail-10.7B-v1.Q5_0.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q5_0.gguf) | Q5_0 | 6.89GB |
33
+ | [LMCocktail-10.7B-v1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q5_K_S.gguf) | Q5_K_S | 6.89GB |
34
+ | [LMCocktail-10.7B-v1.Q5_K.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q5_K.gguf) | Q5_K | 7.08GB |
35
+ | [LMCocktail-10.7B-v1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q5_K_M.gguf) | Q5_K_M | 7.08GB |
36
+ | [LMCocktail-10.7B-v1.Q5_1.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q5_1.gguf) | Q5_1 | 7.51GB |
37
+ | [LMCocktail-10.7B-v1.Q6_K.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q6_K.gguf) | Q6_K | 8.2GB |
38
+ | [LMCocktail-10.7B-v1.Q8_0.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q8_0.gguf) | Q8_0 | 10.62GB |
39
+
40
+
41
+
42
+
43
+ Original model description:
44
+ ---
45
+ license: llama2
46
+ ---
47
+
48
+ # LM-cocktail 10.7B v1
49
+
50
+
51
+ This is a 50%-50% model of the SOLAR model and meow.
52
+
53
+ https://huggingface.co/upstage/SOLAR-10.7B-Instruct-v1.0
54
+
55
+ https://huggingface.co/rishiraj/meow
56
+
57
+
58
+ who rank #1 and #2 among models <13B in the https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard by 2023/12/20.
59
+
60
+ # Alpaca Eval
61
+
62
+ I am thrilled to announce that ChatGPT has ranked LMCocktail 10.7B as the second best model next to GPT4 on AlpcaEval in my local community run. You can also check the leaderboard at [./alpaca_eval/chatgpt_fn_--SOLAR-10-7B-LMCocktail/](./alpaca_eval/chatgpt_fn_--SOLAR-10-7B-LMCocktail/)
63
+
64
+ ```
65
+ win_rate standard_error n_total avg_length
66
+ gpt4 73.79 1.54 805 1365
67
+ SOLAR-10.7B-LMCocktail(new)73.45 1.56 804 1203
68
+ claude 70.37 1.60 805 1082
69
+ chatgpt 66.09 1.66 805 811
70
+ wizardlm-13b 65.16 1.67 805 985
71
+ vicuna-13b 64.10 1.69 805 1037
72
+ guanaco-65b 62.36 1.71 805 1249
73
+ oasst-rlhf-llama-33b 62.05 1.71 805 1079
74
+ alpaca-farm-ppo-human 60.25 1.72 805 803
75
+ falcon-40b-instruct 56.52 1.74 805 662
76
+ text_davinci_003 50.00 0.00 805 307
77
+ alpaca-7b 45.22 1.74 805 396
78
+ text_davinci_001 28.07 1.56 805 296
79
+ ```
80
+
81
+
82
+ # Code
83
+
84
+ The LM-cocktail is novel technique for merging multiple models https://arxiv.org/abs/2311.13534
85
+
86
+ Code is backed up by this repo https://github.com/FlagOpen/FlagEmbedding.git
87
+
88
+ Merging scripts available under the [./scripts](./scripts) folder
89
+
90
+
91
+ # Result
92
+
93
+ The SOLAR model is the first model <30B that can answer this question from my test:
94
+
95
+ ```
96
+ What will AI be like in the year 1010 A.D?
97
+ ```
98
+
99
+ without hullicinating into 1010 A.D is a future time (like other llama2 models)
100
+
101
+ Models greater than that, like Yi-34B could answer this paradoxic question correctly as well, since it is huge enough.
102
+
103
+ ### SOLAR 10.7B output
104
+
105
+ ![img](./assets/SOLAR.png)
106
+
107
+ ### LMCocktail 10.7B output1
108
+
109
+ ![img](./assets/SOLAR_mixed.png)
110
+
111
+ ### LMCocktail 10.7B output2
112
+
113
+ ![img](./assets/SOLAR_mixed2.png)
114
+