xtuner

/

llava-llama-3-8b-v1_1

like 107

I have attempted running llava-surgery-v2.py and come up against this error:

I have checked your config and see this line:

Am I to understand that this is not, in fact, a Llava model?

this is a regular llama 3 that has never seen multimodality?

yea this pretty much confirms it.

I have succeeded in converting the projector to GGUF by first converting the safetensors to bin. Unfortunately the mmproj output by this process crashed on load in both llamacpp and koboldcpp. Please advise.

@jeiku Hi!
We have released weights similar to LLaVA v1.5/v1.6 architecture here. You can try this model with your workflow.

At the same time, our team is also working on the gguf conversion. So we really look forward to your feedback!

Thanks again!

I found that maybe we should follow the LLaVA-v1.6's conversion steps and use llava-surgery-v2.py, because this model uses LoRA finetuning on clip-vit.

https://github.com/ggerganov/llama.cpp/blob/master/examples/llava/README.md#llava-16-gguf-conversion

I found that maybe we should follow the LLaVA-v1.6's conversion steps and use llava-surgery-v2.py, because this model uses LoRA finetuning on clip-vit.

https://github.com/ggerganov/llama.cpp/blob/master/examples/llava/README.md#llava-16-gguf-conversion

Following the 1.6, method I am left with a mmproj file which shows this error after image upload:

Following the 1.5 method, I am left with an mmproj which works, but has no stop token and generates 500 tokens of text every time.

Can you please upload a working mmproj if you have produced one.

@jeiku Thanks very much for your effort!

We are currently developing llava-phi-3-mini models. After the release of the new llava models (expected in 1-2 days), we will proceed with the conversion to gguf.

Hi!@jeiku

GGUF model is here.

https://huggingface.co/xtuner/llava-llama-3-8b-v1_1-gguf