Predicting Forward & Backward Facial Depth Maps From a Single RGB Image For Mobile 3d AR Application Cheap and fast 3D asset creation to enable ARVR applications is a fast growing domain. This paper addresses a significant problem of reconstructing complete 3D information of a face in near realtime speed on a mobile phone. We propose a novel deep learning based solution to predict robust depth maps of a face, one forward facing and the other backward facing, from a single image from the wild. A critical contribution is that the proposed network is capable of learning the depths of the occluded part of the face too. This is achieved by training a fully convolutional neural network to learn the dual (forward and backward) depth maps, with a common encoder and two separate decoders. The 300WLP, a cloud point dataset, is used to compute the required dual depth maps from the training data. The code and results will be made available at project page.

Full Resolution Dense Depth Recovery by Fusing RGB Images and Sparse Depth Full resolution depth is required in many real world engineering applications. However, exist depth sensors only offer sparse depth sample points with limited resolution and noise, e.g., LiDARs. We here propose a deep learning based full resolution depth recovery method from monocular images and corresponding sparse depth measurements of target environment. The novelty of our idea is that the structure similar information between the RGB image and depth image is used to refine the dense depth estimation result. This important similar structure information can be found using a correlation layer in the regression neural network. We show that the proposed method can achieve higher estimation accuracy compared to the state of the art methods. The experiments conducted on the NYU Depth V2 prove the novelty of our idea.