SketchMetaFace: A Learning-based Sketching Interface for High-fidelity
  3D Character Face Modeling
Authors: name: Zhongjin Luo; name: Dong Du; name: Heming Zhu; name: Yizhou Yu; name: Hongbo Fu; name: Xiaoguang Han
Published: 2023-07-03 07:41:07

Abstract: Modeling 3D avatars benefits various application scenarios such as AR/VR,
gaming, and filming. Character faces contribute significant diversity and
vividity as a vital component of avatars. However, building 3D character face
models usually requires a heavy workload with commercial tools, even for
experienced artists. Various existing sketch-based tools fail to support
amateurs in modeling diverse facial shapes and rich geometric details. In this
paper, we present SketchMetaFace - a sketching system targeting amateur users
to model high-fidelity 3D faces in minutes. We carefully design both the user
interface and the underlying algorithm. First, curvature-aware strokes are
adopted to better support the controllability of carving facial details.
Second, considering the key problem of mapping a 2D sketch map to a 3D model,
we develop a novel learning-based method termed "Implicit and Depth Guided Mesh
Modeling" (IDGMM). It fuses the advantages of mesh, implicit, and depth
representations to achieve high-quality results with high efficiency. In
addition, to further support usability, we present a coarse-to-fine 2D
sketching interface design and a data-driven stroke suggestion tool. User
studies demonstrate the superiority of our system over existing modeling tools
in terms of the ease to use and visual quality of results. Experimental
analyses also show that IDGMM reaches a better trade-off between accuracy and
efficiency. SketchMetaFace are available at
https://zhongjinluo.github.io/SketchMetaFace/.