What are you planning for the next release?
Can you say what is planned for the next model update? From my opinion is worth considering to fix some major issues: identity drift in realistic longer videos, better eyes and teeth in full body, better support for 720p
Nice model btw
I agree. Same with dance where the face is not vissible for a while for example: turning around, the identity drift then. The model lacks proper separation between the video reference and the character identity from the photo reference.
Probably the datasets Viggle authors used has low amount of half/full body shot. Side to side real video vs Viggle-Animate it's clear you can say which one is AI due of the issues with face, I think next thing that pushes the issue further is low resolution. Can someone from dev team let us know if there's any chance to fix it in next release?
Thank you for the feedback. We are aware of these limitations and are actively working toward the following improvements:
- Lip-sync and eye movement; facial identity preservation remains a fundamental challenge for current video generation models
- Longer clips, multi-shot sequences, and higher resolution
- Multi-person replacement
- Background replacement
These are long-term objectives with no guaranteed timeline, but we remain committed to pushing forward.
Please note that Viggle-Animate is not designed as a comprehensive video editing solution. Tasks such as motion editing for actors and voice manipulation fall outside the scope of this project.
We are currently preparing training data and will release updates upon achieving substantial improvements.
Thank you for the feedback. We are aware of these limitations and are actively working toward the following improvements:
- Lip-sync and eye movement; facial identity preservation remains a fundamental challenge for current video generation models
- Longer clips, multi-shot sequences, and higher resolution
- Multi-person replacement
- Background replacement
These are long-term objectives with no guaranteed timeline, but we remain committed to pushing forward.
Please note that Viggle-Animate is not designed as a comprehensive video editing solution. Tasks such as motion editing for actors and voice manipulation fall outside the scope of this project.
We are currently preparing training data and will release updates upon achieving substantial improvements.
I tested various scenarios today, and it’s clear that identity drift is virtually non-existent in close-ups, however, the model struggles significantly with movement in half or full-body shots, or when the subject moves toward or away from the camera. Similarly during a body rotation — even if the face is hidden for just half a second—the model might copy the face from the video when it's vissible again instead of maintaining the face from the source photo. I hope these kinds of details are helpful.
Yes please more half and full body shots in training data