LLama-cpp-python compatibility
#7
by
enrico07
- opened
Is LLava-v1.6 34B compatible with LLama-cpp-python library? If so, should I use the same code that I would use for LLava-v1.5 family or should I use a different prompt format?
I don't know about the library, it's probably wrapping llama.cpp so it is compatible to inference the llava-1.6 models.
However, whatever code/functions you use to inference it, they need to access the new functionality.
llava-1.6 splits images into grids, created embeddings and permutes those into grids.
llama.cpp -> clip.cpp/llava.cpp those now contains the necessary APIs. You can see them being using in llava-cli.cpp