Is it possible to quantise this model using GGUF?

by shrikrishnaholla - opened Aug 23

Aug 23

I am interested in using this using Ollama. However, GGUF my Repo doesn't seem to be able to detect / convert it. Has anyone tried to quantise this model? How would one approach it?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment