Update README.md
Browse files
README.md
CHANGED
|
@@ -1,6 +1,39 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: other
|
| 3 |
-
license_name: nvidia-open-model-license
|
| 4 |
-
license_link: >-
|
| 5 |
-
https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
|
| 6 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: nvidia-open-model-license
|
| 4 |
+
license_link: >-
|
| 5 |
+
https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/
|
| 6 |
+
---
|
| 7 |
+
# Audio2Face-3D
|
| 8 |
+
|
| 9 |
+
## Description
|
| 10 |
+
**Audio2Face-3D** leverages state-of-the-art deep learning techniques to transform audio input into highly detailed facial animations. By utilizing a high-quality 4D capture dataset and sophisticated network architectures, our system can produce realistic facial animations of skin, teeth, tongue, and eyeballs. The system supports real-time interaction, making it suitable for both live applications and offline facial animation authoring.
|
| 11 |
+
|
| 12 |
+
**Model Developer**: NVIDIA
|
| 13 |
+
|
| 14 |
+
## Model Versions
|
| 15 |
+
The Audio2Face-3D release includes
|
| 16 |
+
* [Audio2Face-3D-v3.0](https://huggingface.co/nvidia/Audio2Face-3D-v3.0) (diffusion-based network for multiple identities)
|
| 17 |
+
* [Audio2Face-3D-v2.3-Mark](https://huggingface.co/nvidia/Audio2Face-3D-v2.3-Mark) (regression-based network for Mark identity)
|
| 18 |
+
* [Audio2Face-3D-v2.3-Claire](https://huggingface.co/nvidia/Audio2Face-3D-v2.3-Claire) (regression-based network for Claire identity)
|
| 19 |
+
* [Audio2Face-3D-v2.3-James](https://huggingface.co/nvidia/Audio2Face-3D-v2.3-James) (regression-based network for James identity)
|
| 20 |
+
|
| 21 |
+
Note, all networks receive common inputs of audio and emotion labels and output motion deltas for facial skin, tongue, jaw, and eyeballs.
|
| 22 |
+
|
| 23 |
+
## Correspondence to
|
| 24 |
+
Yeongho Seol (yseol@nvidia.com), Michael Huang (michaelhu@nvidia.com)
|
| 25 |
+
|
| 26 |
+
|
| 27 |
+
## License
|
| 28 |
+
Your use of this model is governed by the [NVIDIA Open Model License](https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-license/).
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
## Citation
|
| 32 |
+
```
|
| 33 |
+
@article{chung2025audio2face,
|
| 34 |
+
title={Audio2Face-3D: Audio-driven Realistic Facial Animation For Digital Avatars},
|
| 35 |
+
author={Chung, Chaeyeon and Fedorov, Ilya and Huang, Michael and Karmanov, Aleksey and Korobchenko, Dmitry and Ribera, Roger and Seol, Yeongho},
|
| 36 |
+
journal={arXiv preprint arXiv:00000000},
|
| 37 |
+
year={2025}
|
| 38 |
+
}
|
| 39 |
+
```
|