  This work focuses on complete 3D facial geometry prediction, including 3D
facial alignment via 3D face modeling and face orientation estimation using the
proposed multi-task, multi-modal, and multi-representation landmark refinement
network (M$^3$-LRN). Our focus is on the important facial attributes, 3D
landmarks, and we fully utilize their embedded information to guide 3D facial
geometry learning. We first propose a multi-modal and multi-representation
feature aggregation for landmark refinement. Next, we are the first to study
3DMM regression from sparse 3D landmarks and utilize multi-representation
advantage to attain better geometry prediction. We attain the state of the art
from extensive experiments on all tasks of learning 3D facial geometry. We
closely validate contributions of each modality and representation. Our results
are robust across cropped faces, underwater scenarios, and extreme poses.
Specially we adopt only simple and widely used network operations in M$^3$-LRN
and attain a near 20\% improvement on face orientation estimation over the
current best performance. See our project page here.
