Post
1861
How about engaging in a creative chat with your favorite video character? 💬
@chansung and I worked on a weekend project combining the benefits of Gemini 1.0 and powerful chat models like Zephyr to demo this.
We use Gemini 1.0 to produce the personality traits of any character found in an input video. We then prepare a system prompt with the discovered traits to start chatting with an LLM (Zephyr in this case).
Managing a video captioning model is a little out of our expertise, hence Gemini FTW here 😶🌫️
👨💻 Code: https://github.com/deep-diver/Vid2Persona
🤗 Demo: chansung/vid2persona
@chansung and I worked on a weekend project combining the benefits of Gemini 1.0 and powerful chat models like Zephyr to demo this.
We use Gemini 1.0 to produce the personality traits of any character found in an input video. We then prepare a system prompt with the discovered traits to start chatting with an LLM (Zephyr in this case).
Managing a video captioning model is a little out of our expertise, hence Gemini FTW here 😶🌫️
👨💻 Code: https://github.com/deep-diver/Vid2Persona
🤗 Demo: chansung/vid2persona