I'd like to see a GGUF version of this
#5
by
iplayfast
- opened
so it can be run with ollama
you can read my mind, will be great yes
Unless someone beats me to it, I might take a stab at this and also quantizing it, this weekend. Don't know if it's possible, but I'll give it a shot.
I don't know what to do with the vision model, though. Does anyone have more info on that? I've extracted the KV from it, but not sure how to proceed after that.
Any update yet about the GGUF?