Papers
arxiv:2309.02186

AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collections

Published on Sep 5, 2023
· Submitted by akhaliq on Sep 6, 2023
#2 Paper of the day
Authors:
,
,
,
,
,
,

Abstract

Previous animatable 3D-aware GANs for human generation have primarily focused on either the human head or full body. However, head-only videos are relatively uncommon in real life, and full body generation typically does not deal with facial expression control and still has challenges in generating high-quality results. Towards applicable video avatars, we present an animatable 3D-aware GAN that generates portrait images with controllable facial expression, head pose, and shoulder movements. It is a generative model trained on unstructured 2D image collections without using 3D or video data. For the new task, we base our method on the generative radiance manifold representation and equip it with learnable facial and head-shoulder deformations. A dual-camera rendering and adversarial learning scheme is proposed to improve the quality of the generated faces, which is critical for portrait images. A pose deformation processing network is developed to generate plausible deformations for challenging regions such as long hair. Experiments show that our method, trained on unstructured 2D images, can generate diverse and high-quality 3D portraits with desired control over different properties.

Community

Uploading yk558_In_a_coastal_city__under_the_soft_lighting_of_a_setting_s_f21815a3-b94f-4e48-8536-8614787c78ed.png…

Uploading yk558_In_a_coastal_city__under_the_soft_lighting_of_a_setting_s_b7a3d134-ff54-43b0-953e-46ab442c0072.png…

Default_Russia_Russian_version_Girl_of_Slavic_appearance_beaut_0_5e5e8751-6bb7-4343-9f2e-72dcf48752e3_1.jpg

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2309.02186 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2309.02186 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2309.02186 in a Space README.md to link it from this page.

Collections including this paper 8