Hugging Face's logo

Spaces:
ggml-org
/
gguf-my-repo
Running on A10G

App Files Community
95

Cannot Quantize Llama3-70B models

#92
by xi0v - opened
Discussion
xi0v

I keep getting this error "Connection errored out" when trying to quantize one of my llama3-70B models. Is there a model size limit?

xi0v

@reach-vb I apologize for the ping but can you please look into this?

reach-vb
 ggml.ai org

Hi @xi0v - yes, sorry for the delay in response and thanks for the ping. Yes! the issue is that we only have 60GB data disk attached at the moment. I plan on fixing it soon.

reach-vb changed discussion status to closed

Sign up or log in to comment