add-quantized-gguf-files
#1
by
espenhk
- opened
No description provided.
Same as with the regular Mistral and Llama-models, this PR will add quantized GGUF versions of the model s.t. you can run it on your own machine (for instance using Ollama). No particular config choices were made, other than using f16 vectors (matches previous, as well as the safetensors files).
Ready on my end!
espenhk
changed pull request status to
open
NorLLM-NTNU
changed pull request status to
merged