  This paper proposes a simple method which solves an open problem of
multi-view 3D-Reconstruction for objects with unknown and generic surface
materials, imaged by a freely moving camera and a freely moving point light
source. The object can have arbitrary (e.g. non-Lambertian), spatially-varying
(or everywhere different) surface reflectances (svBRDF). Our solution consists
of two smallsized neural networks (dubbed the 'Shape-Net' and 'BRDFNet'), each
having about 1,000 neurons, used to parameterize the unknown shape and unknown
svBRDF, respectively. Key to our method is a special network design (namely, a
ResNet with a global feedback or 'ring' connection), which has a provable
guarantee for finding a valid diffeomorphic shape parameterization. Despite the
underlying problem is highly non-convex hence impractical to solve by
traditional optimization techniques, our method converges reliably to high
quality solutions, even without initialization. Extensive experiments
demonstrate the superiority of our method, and it naturally enables a wide
range of special-effect applications including novel-view-synthesis,
relighting, material retouching, and shape exchange without additional coding
effort. We encourage the reader to view our demo video for better
visualizations.
