File size: 8,005 Bytes
e05b6be |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 |
---
license: apache-2.0
pipeline_tag: text-to-video
---
# ID-Animator
This repository is the official checkpoint of [ID-Animator](https://id-animator.github.io/).
It is a Zero-shot ID-Preserving Human Video Generation framework. It can generate high-quality ID-specific human video with only one ID image as reference.
**[ID-Animator: Zero-Shot Identity-Preserving Human Video Generation](https://id-animator.github.io/)**
</br>
[Xuanhua He](https://scholar.google.com/citations?user=-bDAN2YAAAAJ&hl=en&oi=ao),
[Quande Liu*](https://liuquande.github.io/),
[Shengju Qian](https://scholar.google.com/citations?user=QNnWmasAAAAJ&hl=zh-CN),
Xin Wang,
Tao Hu,
[Ke Cao](https://scholar.google.com/citations?user=3qMrWmgAAAAJ&hl=en&oi=ao),
Keyu Yan,
Jie Zhang*
(*Corresponding Author)
[![arXiv](https://img.shields.io/badge/arXiv-2404.15275-b31b1b.svg)](https://arxiv.org/abs/2404.15275)
[![Project Page](https://img.shields.io/badge/Project-Website-green)](https://id-animator.github.io/)
[![Online Demo](https://img.shields.io/badge/%F0%9F%A4%97online%20demo-yellow)](https://huggingface.co/spaces/ID-Animator/ID-Animator)
## Human Video Generation Demos
### Recontextualization
<table class="center">
<tr style="line-height: 0">
<td width=25% style="border: none; text-align: center">Reference Image</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
</tr>
<tr>
<td width=25% style="border: none"><img src="./__assets__/ref/lecun1.png" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/first_part/lecun/2.gif" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/first_part/lecun/3.gif" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/first_part/lecun/4.gif" style="width:100%"></td>
</tr>
</table>
<table class="center">
<tr style="line-height: 0">
<td width=25% style="border: none; text-align: center">Reference Image</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
</tr>
<tr>
<td width=25% style="border: none"><img src="./__assets__/ref/fbb.png" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/first_part/ann/1.gif" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/first_part/ann/4.gif" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/first_part/ann/6.gif" style="width:100%"></td>
</tr>
</table>
### Inference with Community Models
<table class="center">
<tr style="line-height: 0">
<td width=25% style="border: none; text-align: center">Reference Image</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
</tr>
<tr>
<td width=25% style="border: none"><img src="./__assets__/ref/hinton.png" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/second/hinton/2.gif" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/second/hinton/3.gif" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/second/hinton/6.gif" style="width:100%"></td>
</tr>
</table>
<table class="center">
<tr style="line-height: 0">
<td width=25% style="border: none; text-align: center">Reference Image</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
</tr>
<tr>
<td width=25% style="border: none"><img src="./__assets__/ref/taylor.png" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/second/taylor/4.gif" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/second/taylor/5.gif" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/second/taylor/6.gif" style="width:100%"></td>
</tr>
</table>
### Identity Mixing
<table class="center">
<tr style="line-height: 0">
<td width=25% style="border: none; text-align: center">Reference Image 1</td>
<td width=25% style="border: none; text-align: center">Reference Image 2</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
</tr>
<tr>
<td width=25% style="border: none"><img src="./__assets__/ref/cl.png" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/ref/sms.png" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/third/1/1.gif" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/third/1/6.gif" style="width:100%"></td>
</tr>
</table>
<table class="center">
<tr style="line-height: 0">
<td width=25% style="border: none; text-align: center">Reference Image 1</td>
<td width=25% style="border: none; text-align: center">Reference Image 2</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
</tr>
<tr>
<td width=25% style="border: none"><img src="./__assets__/ref/sansa.png" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/ref/musk.png" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/third/2/2.gif" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/third/2/6.gif" style="width:100%"></td>
</tr>
</table>
### Combination with ControlNet
<table class="center">
<tr style="line-height: 0">
<td width=25% style="border: none; text-align: center">Reference Image</td>
<td width=25% style="border: none; text-align: center">Sketch Image</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
</tr>
<tr>
<td width=25% style="border: none"><img src="./__assets__/ref/fbb.png" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/ref/sketch.png" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/fourth/1.gif" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/fourth/2.gif" style="width:100%"></td>
</tr>
</table>
<table class="center">
<tr style="line-height: 0">
<td width=25% style="border: none; text-align: center">Reference Image</td>
<td width=25% style="border: none; text-align: center">Sketch Sequence</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
<td width=25% style="border: none; text-align: center">Output Video</td>
</tr>
<tr>
<td width=25% style="border: none"><img src="./__assets__/ref/fbb.png" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/ref/sketch_sequence.png" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/fourth/3.gif" style="width:100%"></td>
<td width=25% style="border: none"><img src="./__assets__/fourth/4.gif" style="width:100%"></td>
</tr>
</table>
## Contact Us
**Xuanhua He**: hexuanhua@mail.ustc.edu.cn
**Quande Liu**: qdliu0226@gmail.com
**Shengju Qian**: thesouthfrog@gmail.com
|