TheBloke

/

Mixtral-8x7B-Instruct-v0.1-GGUF

like 562

I love it! My 16GB 4060ti is getting over 5 tokens/s with the Q4_K_M file. Offloaded 12 layers, and it's using another 32GB system RAM.

You're a legend mate!