Our Method Fan et al. openMVS frame 1 frame 5 frame 10 frame 15 3D-R2N2 frame 1 frame 5 frame 10 frame 15 frame 1 frame 5 frame 10 frame 15 frame 1 frame 5 frame 10 frame 15 final output Figure 1: We introduce our approach which recovers both global camera poses and point-cloud-based shape in full 3D from a few observations in limited views, and yet guarantees photometric consistency across views. The subset of the reconstructed point cloud which are occluded from any views in the frame is marked red. In contrast Fan et al.[7] and 3D-R2N2[5] reconstructs shapes from single frames in canonical pose with no cameras, and openMVS[9] produces SfM results with accurate but sparse reconstruction for visible parts only. Reconstructing 3D shapes from a sequence of images has long been a problem of interest in computer vision. Classical Structure from Motion (SfM) methods have attempted to solve this problem through projected point displacement & bundle adjustment. More recently, deep methods have attempted to solve this problem by directly learning a relationship between geometry and appearance. There is, however, a significant gap between these two strategies. SfM tackles the problem from purely a geometric perspective, taking no account of the object shape prior. Modern deep methods more often throw away geometric constraints altogether, rendering the results unreliable. In this paper we make an effort to bring these two seemingly disparate strategies together. We introduce learned shape prior in the form of deep shape generators into Photometric Bundle Adjustment (PBA) and propose to accommodate full 3D shape generated by the shape prior within the optimization-based inference framework, demonstrating impressive results.
Latest in the line of such implementations is REF which recover both global camera pose and 3D point cloud based shape with very few, limited view observations.
Object-Centric Photometric Bundle Adjustment with Deep Shape Prior
2018 IEEE Winter Conference on Applications of Computer Vision (WACV)
2018 IEEE Winter Conference on Applications of Computer Vision (WACV)
Computer Science