How to merge .gguf into one file?
#34
by
Lunitaris
- opened
Hello,
I downloaded the following files:
Mixtral-8x22B-Instruct-v0.1.Q6-00001-of-00004.gguf
Mixtral-8x22B-Instruct-v0.1.Q6-00002-of-00004.gguf
Mixtral-8x22B-Instruct-v0.1.Q6-00003-of-00004.gguf
Mixtral-8x22B-Instruct-v0.1.Q6-00004-of-00004.gguf
And I would like to know how to merge them into one file?
I am using Jan, AnythingLLM, Faraday to run models, they need a single file.
Thx
Hi,
It's very easy, you can follow this tutorial to merge them back together: https://github.com/ggerganov/llama.cpp/discussions/6404
Check this solution: https://huggingface.co/TheBloke/Falcon-180B-Chat-GGUF/discussions/1