metadata
tags:
- text-to-image
- stable-diffusion
- audio-to-video
license: apache-2.0
language:
- en
library_name: diffusers
V-Express Model Card
Introduction
Models
Audio Encoder
- model_ckpts/wav2vec2-base-960h. (It is also available from the original model card facebook/wav2vec2-base-960h)
Face Analysis
- model_ckpts/insightface_models/models/buffalo_l. (It is also available from the original repository insightface/buffalo_l)
V-Express
- model_ckpts/sd-vae-ft-mse. VAE encoder. (original model card stabilityai/sd-vae-ft-mse)
- model_ckpts/stable-diffusion-v1-5. Only the model configuration file for unet is needed here. (original model card runwayml/stable-diffusion-v1-5)
- model_ckpts/v-express. The video generation model conditional on audio and V-kps we call V-Express.
Paper
- arxiv.org/abs/2406.02511