GGUF format?
#3
by
Puchacz19
- opened
Hello, could you please make this model in GGUF format? I only have 24VRAM and 64 RAM memory.
It's already made here:
https://huggingface.co/TheBloke/goliath-120b-GGUF
It's already made here:
https://huggingface.co/TheBloke/goliath-120b-GGUF
Yes, but it;s base model, not "rpcal".
Puchacz19
changed discussion status to
closed
Puchacz19
changed discussion status to
open
It's already made here:
https://huggingface.co/TheBloke/goliath-120b-GGUFYes, but it;s base model, not "rpcal".
It is not possible to do a "rpcal" version for GGUF because that quant method doesn't use a calibration dataset.
Base model is the same Goliath 120B for both normal and rpcal versions, just used a different calibration dataset.
Panchovix
changed discussion status to
closed