Post
🎥 🤾 Vid2Persona: talk to person from video clip
A fun project over the last week with @sayakpaul . It has a simple pipeline from extracting traits of video characters to chatting with them.
Under the hood, this project leverages the power of both commercial and open source models. We used Google's Gemini 1.0 Pro Vision model to understand the video content directly, then we used HuggingFaceH4/zephyr-7b-beta model to make conversation!
Try it Hugging Face Space and let us know what you think.
: chansung/vid2persona
The space application is a dedicated implementation for ZeroGPU environment + Hugging Face Inference API with PRO account. If you wish to host it on your own environment, consider duplicate the space or run locally with the project repository
: https://github.com/deep-diver/Vid2Persona
A fun project over the last week with @sayakpaul . It has a simple pipeline from extracting traits of video characters to chatting with them.
Under the hood, this project leverages the power of both commercial and open source models. We used Google's Gemini 1.0 Pro Vision model to understand the video content directly, then we used HuggingFaceH4/zephyr-7b-beta model to make conversation!
Try it Hugging Face Space and let us know what you think.
: chansung/vid2persona
The space application is a dedicated implementation for ZeroGPU environment + Hugging Face Inference API with PRO account. If you wish to host it on your own environment, consider duplicate the space or run locally with the project repository
: https://github.com/deep-diver/Vid2Persona