RGBtoTSDF: Direct TSDF Prediction from a Single RGB Image for Dense 3D Reconstruction In this paper, we present a novel method to predict 3D TSDF voxels from a single image for dense 3D reconstruction. 3D reconstruction with RGB images has two inherent problems: scale ambiguity and sparse reconstruction. With the advent of deep learning, depth prediction from a single RGB image has addressed these problems. However, as the predicted depth is typically noisy, denoising methods such as TSDF fusion should be adapted for the accurate scene reconstruction. To integrate the twostep processing of depth prediction and TSDF generation, we design an RGBtoTSDF network to directly predict 3D TSDF voxels from a single RGB image. The TSDF using our network can be generated more efficiently in terms of time and accuracy than the TSDF converted from depth prediction. We also use the predicted TSDF for a more accurate and robust camera pose estimation to complete scene reconstruction. The global TSDF is updated from TSDF prediction and pose estimation, and thus dense isosurface can be extracted. In the experiments, we evaluate our TSDF prediction and camera pose estimation results against the conventional method.

Directional TSDF: Modeling Surface Orientation for Coherent Meshes Realtime 3D reconstruction from RGBD sensor data plays an important role in many robotic applications, such as object modeling and mapping. The popular method of fusing depth information into a truncated signed distance function (TSDF) and applying the marching cubes algorithm for mesh extraction has severe issues with thin structures: not only does it lead to loss of accuracy, but it can generate completely wrong surfaces. To address this, we propose the directional TSDFxe2x80x94a novel representation that stores opposite surfaces separate from each other. The marching cubes algorithm is modified accordingly to retrieve a coherent mesh representation. We further increase the accuracy by using surface gradientbased ray casting for fusing new measurements. We show that our method outperforms stateoftheart TSDF reconstruction algorithms in mesh accuracy.