Towards Uncalibrated Photometric Stereo for Non-Lambertian Surfaces

Photometric stereo is a method that captures local shape and reflectance of a 3D object from several intensity images of this object taken under different illuminations and a fixed viewpoint. The ability to estimate the surface reflectance parameters and local shape makes photometric stereo an invaluable tool for capturing intrinsic properties of real surfaces. The greatest benefit from this ability receive mainly augmented reality applications where realistic texture mapping and relighting of virtual objects is widely needed. The recovered reflectance parameters can also be used for visual quality control applications. We are interested in photometric stereo setups in which the illumination conditions used to acquire individual input images are completely unknown except that they are provided by a single point illuminant at infinity. So far, such uncalibrated photometric stereo setup has been understood for Lambertian surfaces only. It is known that the surface parameters can be recovered up to nine degrees of freedom (9dof). In this thesis, we make a step towards answering the following question: If the object reflectance is not Lambertian, what is the ambiguity up to which the local surface parameters can be recovered by uncalibrated photometric stereo? We choose the following approach to answer this question. We identify fundamental properties of reflectance which are shared over large classes of BRDFs. These properties include, for example, symmetries (e.g., the BRDF reciprocity or isotropy). We analyse whether a given reflectance property is able to deliver some information about the surface or light parameters. We approximate an arbitrary surface reflectance by Lambertian reflectance and perform the surface parameters recovery using uncalibrated Lambertian photometric stereo. Subsequently, we use additional information provided by a given BRDF property to reduce the 9dof ambiguity. The main result of this thesis is that we found two fundamental reflectance properties which are useful for this goal. Firstly, we showed that if the surface reflectance is a composition of Lambertian and ideal specular reflectance then the albedo map is recovered up to a global scale factor, and the normal map is recovered up to a rotation around the viewing direction (1dof) and the change in coordinate frame handedness (a binary ambiguity). Subsequent application of the known integrability constraint which requires the normals to correspond to a continuous surface results in further reducing the ambiguity of the normal map determination to only the convex-concave (binary ambiguity). Secondly, we showed that if the BRDF is separable and isotropic then it is possible to construct a measurement which reveals the projections of the normals onto the image plane and onto the plane perpendicular to the light direction. In this case the albedo map is again recovered up to a global scale, and the normals are determined up to a convex-concave (binary) ambiguity without the need of the integrability constraint. These results identify two important cases in which the necessity of knowing the illumination parameters can be relaxed. They are likely to become the basis of even wider use of photometric stereo in practice.