pages/9_LLaVA-NeXT.py

import streamlit as st
from streamlit_extras.switch_page_button import switch_page

st.title("LLaVA-NeXT")

st.success("""[Original tweet](https://twitter.com/mervenoyann/status/1770832875551682563) (March 21, 2024)""", icon="ℹ️")
st.markdown(""" """)

st.markdown("""LLaVA-NeXT is recently merged to 🤗 Transformers and it outperforms many of the proprietary models like Gemini on various benchmarks!🤩   
For those who don't know LLaVA, it's a language model that can take image 💬  
Let's take a look, demo and more in this. 
""")
st.markdown(""" """)

st.image("pages/LLaVA-NeXT/image_1.jpeg", use_column_width=True)
st.markdown(""" """)

st.markdown("""
LLaVA is essentially a vision-language model that consists of ViT-based CLIP encoder, a MLP projection and Vicuna as decoder ✨  
LLaVA 1.5 was released with Vicuna, but LLaVA NeXT (1.6) is released with four different LLMs:  
- Nous-Hermes-Yi-34B  
- Mistral-7B  
- Vicuna 7B & 13B 
""")
st.markdown(""" """)

st.image("pages/LLaVA-NeXT/image_2.jpeg", use_column_width=True)
st.markdown(""" """)

st.markdown("""
Thanks to Transformers integration, it is very easy to use LLaVA NeXT, not only standalone but also with 4-bit loading and Flash Attention 2 💜  
See below on standalone usage 👇 
""")
st.markdown(""" """)

st.image("pages/LLaVA-NeXT/image_3.jpeg", use_column_width=True)
st.markdown(""" """)

st.markdown("""To fit large models and make it even faster and memory efficient, you can enable Flash Attention 2 and load model into 4-bit using bitsandbytes ⚡️ transformers makes it very easy to do this! See below 👇 
""")
st.markdown(""" """)

st.image("pages/LLaVA-NeXT/image_4.jpeg", use_column_width=True)
st.markdown(""" """)

st.markdown("""If you want to try the code right away, here's the [notebook](https://t.co/NvoxvY9z1u).  
Lastly, you can directly play with the LLaVA-NeXT based on Mistral-7B through the demo [here](https://t.co/JTDlqMUwEh) 🤗 
""")
st.markdown(""" """)

st.video("pages/LLaVA-NeXT/video_1.mp4", format="video/mp4")
st.markdown(""" """)

st.info("""
Ressources:  
[LLaVA-NeXT: Improved reasoning, OCR, and world knowledge](https://llava-vl.github.io/blog/2024-01-30-llava-next/) 
by Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, Yong Jae Lee (2024)   
[GitHub](https://github.com/haotian-liu/LLaVA/tree/main)   
[Hugging Face documentation](https://huggingface.co/docs/transformers/model_doc/llava_next)""", icon="📚")


st.markdown(""" """)
st.markdown(""" """)
st.markdown(""" """)
col1, col2, col3 = st.columns(3)
with col1:
    if st.button('Previous paper', use_container_width=True):
        switch_page("Depth Anything")
with col2:
    if st.button('Home', use_container_width=True):
        switch_page("Home")
with col3:
    if st.button('Next paper', use_container_width=True):
        switch_page("Painter")