Working example
Could you please include a working example in Model card? I am having hard time making it work in Google Colab and after 3 hours I am about to give up.
@hermanda hmm could you share your colab? Il see what’s wrong with it
@YaTharThShaRma999
here it is https://colab.research.google.com/drive/1d3pF9MBOCFpQ_KqWnWdfuG1w_6CBWjrK?usp=sharing
Note that I am using revision="refs/pr/2" as the master can't be even downloaded properly.
@hermanda oh it’s not working because transformers doesn’t support llava 1.6(it has a few different things from previous llava models).
Install the llava repository has that provides the code to run it.
@YaTharThShaRma999 thanks, do you think the support of llava 1.6 will be there in future? if so how many weeks do you think it is going to be realistically?
@hermanda
there is already a pull request in transformers thats mostly done but its still a bit experimental.
install transformers from here instead of pip install transformers
https://github.com/huggingface/transformers/pull/29012
and here is the hf version of the model
https://huggingface.co/llava-hf/llava-v1.6-mistral-7b-hf
thank you