llava-hf

/

llava-1.5-7b-hf

like 121

Hi @mohan007
Yes it is not released yet, we will announce the integration once https://github.com/huggingface/transformers/pull/27662 gets merged and should be added in the next release

When is the next release waiting for that

We just merged the PR, you can try to run the model by installing transformers from source:

pip install -q git+https://github.com/huggingface/transformers.git

Have also a look at the google colab demo: https://colab.research.google.com/drive/1qsl6cd2c8gGtEW1xV5io7S8NHh-Cp1TV?usp=sharing

Thanks

We just merged the PR, you can try to run the model by installing transformers from source:

pip install -q git+https://github.com/huggingface/transformers.git

Have also a look at the google colab demo: https://colab.research.google.com/drive/1qsl6cd2c8gGtEW1xV5io7S8NHh-Cp1TV?usp=sharing

Any speeds up from flash attention we could expect than the original inference from original research author repo

Hi @mohan007
We did not compared the performance of our FA2 vs model the original repository, when comparing our native implementation vs their native implementation it led to similar performance, but this might be different when it comes to using FA2.