Loading with ggml llava through llama.cpp (PR)
#1
by
cmp-nct
- opened
This comment has been hidden
cmp-nct
changed discussion title from
Can you provide the full CLIP model, not just the extracted vision part ?
to never mind
cmp-nct
changed discussion status to
closed
cmp-nct
changed discussion title from
never mind
to Loading with ggml llava
@cmp-nct can you let me know how you fixed this issue ?
Sure, I am using ggml llava inference which loads the CLIP model first for conversion, it expects a full model to extract the vision from it but this is already the vision only part.
I wrote a patch: https://github.com/ggerganov/llama.cpp/pull/4172
Anything that looks for "clip_vision_model" is code that expects a full CLIP, in that case you just need to skip the extraction code.
cmp-nct
changed discussion status to
open
cmp-nct
changed discussion title from
Loading with ggml llava
to Loading with ggml llava through llama.cpp (PR)