We present a convolutional network capable of generating images of a previously unseen object from arbitrary viewpoints given a single image of this object. The input to the network is a single image and the desired new viewpoint; the output is a view of the object from this desired viewpoint. The network is trained on renderings of synthetic 3D models. It learns an implicit 3D representation of the object class, which allows it to transfer shape knowledge from training instances to a new object instance. Beside the color image, the network can also generate the depth map of an object from arbitrary viewpoints. This allows us to predict 3D point clouds from a single image, which can be fused into a surface mesh. We experimented with cars and chairs. Even though the network is trained on artificial data, it generalizes well to objects in natural images without any modifications.
Tatarchenko et al. REF developed a convolutional network to synthesize unseen 3D views from a single image and demonstrated the synthesized images can be used them to reconstruct 3D shape.
Single-view to Multi-view: Reconstructing Unseen Views with a Convolutional Network
ArXiv
ArXiv
Computer Science