Some GGUF Quants with iMatrix for : https://huggingface.co/NeverSleep/MiquMaid-v2-70B-DPO
Q3_K_M is here, IQ3_XXS on the way.
For other IQ quants (and possibly better ones than mine), see there : https://huggingface.co/Kooten/MiquMaid-v2-70B-DPO-Imatrix-GGUF
Some benchs with LlamaCPP :
NeverSleep_MiquMaid-v2-70B-DPO-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,Hellaswag,83.75,,400,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-DPO-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,Hellaswag,83.2,,1000,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-DPO-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,Arc-Challenge,55.51839465,,299,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-DPO-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,Arc-Easy,78.24561404,,570,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-DPO-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,MMLU,47.92332268,,313,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-DPO-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,Thruthful-QA,40.88127295,,817,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-DPO-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,Winogrande,78.8477,,1267,2024-02-07 05:40:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,
NeverSleep_MiquMaid-v2-70B-DPO-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,wikitext,4.3328,512,512,2024-02-07 00:00:00,,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,81
NeverSleep_MiquMaid-v2-70B-DPO-b2093-iMat-c32_ch1000-Q3_K_M.gguf,-,wikitext,3.8581,512,512,2024-02-07 00:00:00,70b,Mistral_Medium,32768,,,GGUF,NeverSleep,Nexesenex,655
The Hellaswag might be underestimated by 5-6 points due to recent changes in LlamaCPP.
- Downloads last month
- 72