HeadCraft: Modeling High-Detail Shape Variations for Animated 3DMMs
Authors: name: Artem Sevastopolsky; name: Philip-William Grassal; name: Simon Giebenhain; name: ShahRukh Athar; name: Luisa Verdoliva; name: Matthias Niessner
Published: 2023-12-21 18:57:52

Abstract: Current advances in human head modeling allow to generate plausible-looking
3D head models via neural representations. Nevertheless, constructing complete
high-fidelity head models with explicitly controlled animation remains an
issue. Furthermore, completing the head geometry based on a partial
observation, e.g. coming from a depth sensor, while preserving details is often
problematic for the existing methods. We introduce a generative model for
detailed 3D head meshes on top of an articulated 3DMM which allows explicit
animation and high-detail preservation at the same time. Our method is trained
in two stages. First, we register a parametric head model with vertex
displacements to each mesh of the recently introduced NPHM dataset of accurate
3D head scans. The estimated displacements are baked into a hand-crafted UV
layout. Second, we train a StyleGAN model in order to generalize over the UV
maps of displacements. The decomposition of the parametric model and
high-quality vertex displacements allows us to animate the model and modify it
semantically. We demonstrate the results of unconditional generation and
fitting to the full or partial observation. The project page is available at
https://seva100.github.io/headcraft.