Combining spectral and photometric stereo techniques for reflectance estimation using an RGB digital camera

In this work we describe one experiment investigating the combination of a photometric stereo and a multispectral technique to recover spectral information with an RGB digital camera. A 4-source based-photometric device is used to recover the scene normals and the albedo at each image pixel. We use a multispectral technique based on a linear pseudoinverse method, which directly relates spectral reflectances and RGB camera values. The suggested method uses the recovered albedo for each channel as RGB components to recover the spectral reflectance pixel by pixel. It is possible then to avoid finding different reflectances in different points of the surface due to shadows, as the albedo recovered by photometric stereo techniques is not affected by shadows. We expect that analyzing the difficulties and problems encountered will be useful in applications such us color image reproduction, threedimensional surface analysis and spectral characterization of materials and micro-facets. Introduction There are many techniques which allow us to obtain 3D information about surfaces. These methods can be classified in what Woodham [1] refers to as direct methods and indirect methods. The first methods are those which try to measure distance ranges directly, while the indirect methods attempt to determine distance by measuring parameters calculated from images of the illuminated objects. Shape from photometric stereo was conceived by Woodham in the early eighties and has been extensively studied with both theoretical and experimental approaches [2-5]. All approaches published since then can be classified according to the assumptions the authors make about the surface they are dealing with and the type of problem they want to solve. Most of those algorithms have been proposed in digital imaging, for color reproduction as a function of RGB values provided by the capture device. But, as the object’s color depends on the registered color signal [6], sometimes is more useful to obtain the spectral reflectance for complete object characterization, which is independent of illumination. Multispectral imaging can recover spectral radiance or reflectance for each pixel of a scene of interest [7]. Usually a multispectral system consists of an RGB or monochrome digital camera coupled with a number of wide-band or narrowband color filters [8]. As opposing to conventional imaging devices, they capture illuminant-independent images and allow accurate spectral and colorimetric reproduction of color images for any lighting conditions. Photometric Stereo Fundamentals Two assumptions are made in photometric stereo [9]. The surface is generally flat, parallel to the image plane of the camera, and is lit by a single light source. The coordinate system is chosen so that the image plane coincides with the xy plane and the z-axis coincides with the viewing direction, as it is shown in figure 1. Then the surface can be described by a 2D height function z = S(x,y) and we can define the gradient components for every point on the surface as: ( ) ( ) x y x S y x p ∂ ∂ = , , , ( ) ( ) y y x S y x q ∂ ∂ = , , (1) And the normal unit vector N as: ( ) q p q p 1 , , 1 1