GGUF of this model?
#1
by EllaPriest45 - opened
Hi this looks good but 29gb, no way this is gonna run on 16gb vram + 32gb of ram... can we get a GGUF version?
Thanks
You can't have a gguf of an int8, they are both quant's just different types of quantization.
ggufs of sulpher are here. he also has eros ggufs
https://huggingface.co/vantagewithai/Sulphur-2-Base-GGUF/tree/main
You should be able to run these and the int8's with 16+32 if you have dynamic vram turned on
Ah. Thanks