Quant of a Vision Model - How did you do it?
#7
by
Luni
- opened
Hiya,
what tool did you use to quant this Model?
I currently have an abliterated model but would like to quant it, to use it in Ollama for our stuff and test around.
Sadly llama.cpp still has no quanting support in it, i'd be curious to hear how you did it.
Thanks!
Luni
changed discussion title from
Quant of a Vision Model
to Quant of a Vision Model - How did you do ti?
Luni
changed discussion title from
Quant of a Vision Model - How did you do ti?
to Quant of a Vision Model - How did you do it?
I would also like to learn how?