Hugging Face's logo

bofenghuang
/
vigogne-7b-instruct

Text Generation Transformers PyTorch French llama LLM text-generation-inference
Model card Files Community
5

llama.cpp ggml 4-bit quantized please

#1
by Sam2x - opened
Discussion
Sam2x

Hi,

First thank you for the effort to do this model , i think it will be great to generate a quantized version to test it on cpu!

Best,

bofenghuang
 Owner

Hi @Sam2x ,

Thanks for your interest!

Due to the license of the Meta LLaMA model, the quantized merged model cannot be shared.

I would recommend reading this section for how to quantize it by yourself.

Sign up or log in to comment