Quantization made by Richard Erkhov. [Github](https://github.com/RichardErkhov) [Discord](https://discord.gg/pvy7H8DZMG) [Request more models](https://github.com/RichardErkhov/quant_request) LMCocktail-10.7B-v1 - GGUF - Model creator: https://huggingface.co/Yhyu13/ - Original model: https://huggingface.co/Yhyu13/LMCocktail-10.7B-v1/ | Name | Quant method | Size | | ---- | ---- | ---- | | [LMCocktail-10.7B-v1.Q2_K.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q2_K.gguf) | Q2_K | 3.73GB | | [LMCocktail-10.7B-v1.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.IQ3_XS.gguf) | IQ3_XS | 4.14GB | | [LMCocktail-10.7B-v1.IQ3_S.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.IQ3_S.gguf) | IQ3_S | 4.37GB | | [LMCocktail-10.7B-v1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q3_K_S.gguf) | Q3_K_S | 4.34GB | | [LMCocktail-10.7B-v1.IQ3_M.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.IQ3_M.gguf) | IQ3_M | 4.51GB | | [LMCocktail-10.7B-v1.Q3_K.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q3_K.gguf) | Q3_K | 4.84GB | | [LMCocktail-10.7B-v1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q3_K_M.gguf) | Q3_K_M | 4.84GB | | [LMCocktail-10.7B-v1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q3_K_L.gguf) | Q3_K_L | 5.26GB | | [LMCocktail-10.7B-v1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.IQ4_XS.gguf) | IQ4_XS | 5.43GB | | [LMCocktail-10.7B-v1.Q4_0.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q4_0.gguf) | Q4_0 | 5.66GB | | [LMCocktail-10.7B-v1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.IQ4_NL.gguf) | IQ4_NL | 5.72GB | | [LMCocktail-10.7B-v1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q4_K_S.gguf) | Q4_K_S | 5.7GB | | [LMCocktail-10.7B-v1.Q4_K.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q4_K.gguf) | Q4_K | 6.02GB | | [LMCocktail-10.7B-v1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q4_K_M.gguf) | Q4_K_M | 6.02GB | | [LMCocktail-10.7B-v1.Q4_1.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q4_1.gguf) | Q4_1 | 6.27GB | | [LMCocktail-10.7B-v1.Q5_0.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q5_0.gguf) | Q5_0 | 6.89GB | | [LMCocktail-10.7B-v1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q5_K_S.gguf) | Q5_K_S | 6.89GB | | [LMCocktail-10.7B-v1.Q5_K.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q5_K.gguf) | Q5_K | 7.08GB | | [LMCocktail-10.7B-v1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q5_K_M.gguf) | Q5_K_M | 7.08GB | | [LMCocktail-10.7B-v1.Q5_1.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q5_1.gguf) | Q5_1 | 7.51GB | | [LMCocktail-10.7B-v1.Q6_K.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q6_K.gguf) | Q6_K | 8.2GB | | [LMCocktail-10.7B-v1.Q8_0.gguf](https://huggingface.co/RichardErkhov/Yhyu13_-_LMCocktail-10.7B-v1-gguf/blob/main/LMCocktail-10.7B-v1.Q8_0.gguf) | Q8_0 | 10.62GB | Original model description: --- license: llama2 --- # LM-cocktail 10.7B v1 This is a 50%-50% model of the SOLAR model and meow. https://huggingface.co/upstage/SOLAR-10.7B-Instruct-v1.0 https://huggingface.co/rishiraj/meow who rank #1 and #2 among models <13B in the https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard by 2023/12/20. # Alpaca Eval I am thrilled to announce that ChatGPT has ranked LMCocktail 10.7B as the second best model next to GPT4 on AlpcaEval in my local community run. You can also check the leaderboard at [./alpaca_eval/chatgpt_fn_--SOLAR-10-7B-LMCocktail/](./alpaca_eval/chatgpt_fn_--SOLAR-10-7B-LMCocktail/) ``` win_rate standard_error n_total avg_length gpt4 73.79 1.54 805 1365 SOLAR-10.7B-LMCocktail(new)73.45 1.56 804 1203 claude 70.37 1.60 805 1082 chatgpt 66.09 1.66 805 811 wizardlm-13b 65.16 1.67 805 985 vicuna-13b 64.10 1.69 805 1037 guanaco-65b 62.36 1.71 805 1249 oasst-rlhf-llama-33b 62.05 1.71 805 1079 alpaca-farm-ppo-human 60.25 1.72 805 803 falcon-40b-instruct 56.52 1.74 805 662 text_davinci_003 50.00 0.00 805 307 alpaca-7b 45.22 1.74 805 396 text_davinci_001 28.07 1.56 805 296 ``` # Code The LM-cocktail is novel technique for merging multiple models https://arxiv.org/abs/2311.13534 Code is backed up by this repo https://github.com/FlagOpen/FlagEmbedding.git Merging scripts available under the [./scripts](./scripts) folder # Result The SOLAR model is the first model <30B that can answer this question from my test: ``` What will AI be like in the year 1010 A.D? ``` without hullicinating into 1010 A.D is a future time (like other llama2 models) Models greater than that, like Yi-34B could answer this paradoxic question correctly as well, since it is huge enough. ### SOLAR 10.7B output ![img](./assets/SOLAR.png) ### LMCocktail 10.7B output1 ![img](./assets/SOLAR_mixed.png) ### LMCocktail 10.7B output2 ![img](./assets/SOLAR_mixed2.png)