GGUF?
#1
by
Vezora
- opened
is there any chance of you doing a gguf of the code llama open code interpreter?
(I would use the deep seek but the 100k context length on Code Llama is more useful than the performance loss, to hook up task weaver and LMstudio.)
There's an issue with a mismatched vocab size for the model. I tried to do a GGUF quant when it was released, but this is the error I get:
Exception: Vocab size mismatch (model has 32000, but /models/OpenCodeInterpreter-CL-34B has 32004).
Unfortunately, no google results for a fix and nothing obvious fixes the issue.
:(. ! I see! Thank you, for trying!!!!