Dense Estimation of Surface Reflectance Parameters
by Selecting Optimum Illumination Conditions
Takashi Machida, Haruo Takemura and Naokazu Yokoya
Graduate School of Information Science, Nara Institute of Science and Technology
8916-5 Takayama, Ikoma, Nara 630-0101, Japan
E-mail: taka-ma, takemura, yokoya@is.aist-nara.ac.jp
Abstract
To reproduce a real object in CG, a texture method
which uses a real image has been widely investigated. In
this method, when the object is represented as a virtual object, the object sometimes looks unnatural since the real
image is influenced by a real illumination condition. Thus,
it is necessary to estimate reflectance properties of object
surfaces. Some researchers have been attempted to solve
this problem using range and texture images. However,since the specular reflection component which is one of thesurface reflectance properties cannot always be observed,
an assumption with a shape and a surface property of theobject is needed. This paper describes a new method ofdensely estimating non-uniform surface reflectance properties of real objects with convex and concave surfaces using registered range and surface color texture images obtained by a laser rangefinder. The proposed method determines positions of light to take color images for discriminating diffuse and specular reflection components of surface reflection. The Torrance-Sparrow model is employed
to estimate reflectance parameters by using color images
under multiple illumination conditions. Experiments show
the usefulness of the proposed method.

1

Introduction

Image based rendering (IBR) methods often have beenused to reproduce real objects in computer graphics (CG).
Since a real image of an object is used as a texture of theobject in IBR, a problem occurs, i.e., the appearance ofthe object is not reproduced appropriately when lighting
conditions of real and CG environments are not consistent.
To overcome the problem, a number of methods of estimating reflectance properties of an object surface havebeen developed [1, 3, 4, 5, 6, 7, 8]. In these methods, reflection models with several parameters are employed and£ Presently with Cybermedia Center, Osaka University.

shape and color information of the object is used to estimate the reflectance parameters.
In some works [1, 3, 7], it is assumed that an object
has uniform reflectance properties over the entire surface.
Reflectance parameters are estimated by using the standard least-squares method with a color image. Due to theassumption, these methods cannot be applied to objects
which consist of several different materials and have nonuniform reflectance properties. To treat non-uniform surface objects, Kay and Caelli [4] have proposed a method
which uses multiple images of an object under different
lighting conditions and estimates reflectance parameters bysolving simultaneous equations. However, the method still
has a problem that results are not stable especially when
the specular reflection component, which is one of the reflection component, is very small.
Recently, Sato et al. [8] have developed a methodology
to estimate non-uniform reflectance properties. They set
up an object on a robot arm and measure the object witha CCD camera and a rangefinder from a large number ofviewpoints by rotating the robot arm. In the method, reflectance parameters are stably acquired by decomposing
the surface reflection into two components based on the
singular value decomposition (SVD) technique. Although
the method can be applied to objects with non-uniform reflectance properties, the shape of an object should be limited to convex. This is because it is difficult to observe
the specular reflection component over the surface, since
the lighting condition for a pose of the object against thecamera cannot be changed in the method.
We propose a new method for estimating non-uniform
reflectance properties by observing the specular reflection component densely in the image obtained by a laser
rangefinder which accurately takes registered range andcolor images of an object. In this paper, an algorithm is
proposed to determine light positions with which the specular reflection component is strongly observed over the surface of a complex object with convex and concave surfaces.

A. Measurement

Normal vector N
Light source
vector L
θdRange data
B. Calculation of surface normal
Possible light position 1
Ratio of observing eachreflection component. . . .

Range of observing
the diffuse reflection

Possible light position M
Ratio of observing eachreflection component

Object surface(a) Diffuse reflection

C. Selection of light source position. . . .

Estimation of reflectance parameters fromrange and color images

2.1 Overview
In the present work, the Torrance-Sparrow model is employed as a surface reflectance model to estimate object
surface reflectance properties from range and color images.
Figure 1 shows a flow diagram of estimating surface reflectance properties. Our process consists of four parts,
which are a measurement of an object (A, D), a preprocessing (B), a selection of light source (C), and an estimationof reflectance property (E).

2.2 Torrance-Sparrow model
In this paper, the Torrance-Sparrow model [9], which
accurately represents object reflectance properties, is employed to estimate reflectance parameters. The TorranceSparrow model is given as:
(1)











(b)Specular reflectionÎ  Ä¼¾



(2)



(3)
(4)



where  represents an observed intensity,  and  denotethe diffuse and specular reflection components,  is an attenuation efficient concerning the distance between a point
light source and an object surface point,   is the diffuse
reflectance parameter,  is the specular reflectance parameter, is the surface roughness parameter which is thestandard deviation of a Gaussian distribution, is a light

Ä

Æ

Î
Ä

source vector,
is a surface normal, is a viewing vector, and ¼ is a reflection vector which is mirrored against
and is given by Equation (4), respectively. All vectors
are unit vectors. Figure 2 illustrates the geometry for thismodel.
In the case of color image, ,  ,  ,  and  consist of
RGB channels and this model is applied to each channel.
In order to estimate reflectance parameters  ,  and , itis necessary to obtain the other parameters ,  , , andat each point of the object surface.

ÆFigure 1. Flow diagram of estimating surface
reflectance properties.

       Æ Ä ÆÎ  
Ä¼  Æ ÄÆ Ä

Object surface

Selected light position m
Surface texture image

E. Estimation of reflectance property

2

Range of observing
the specular reflection

Figure 2. Diffuse and Specular reflection on
an object surface.

D. Measurement
Selected light positoin 1
Surface texture image

Normal Viewing
Light source vector N vector V
vector Lθdθv
Reflection
θr vector L’

Ä

Æ

ÄÎ

2.3 Measurement of an object
To obtain unknown parameters other than reflectanceparameters in the Torrance-Sparrow model, we use a laser
rangefinder (Cyberware 3030RGB) with known positionsof point light sources and a camera for acquiring surface color images, as illustrated in Figure 3(a). Figure3(b) shows its motion during measurement and Figure 3(c)shows the illustration viewed from the top of the device.
This system can obtain registered range and surface color
texture images at the same time by rotating the rangefinder
and the camera around an object, so that there is no registration error, even when an object is measured many times.
By using accurately registered multiple color images, reflectance parameters can be estimated accurately by solving simultaneous equations. In Figure 3(c), a camera islocated at  and a texture image is acquired through mirrors which are located at  and . We assume that thecamera is located at  virtually and the camera looks toward the center of rotation.

2.4 Preprocessing
Generally, the noise and quantization error are includedin the range image acquired from the laser rangefinder.
There is also a problem that the surface normal is not calculated accurately around the discontinuity in the range image. Therefore, we employ local quadratic surface fitting.
Firstly, the 55 median filter is applied to the range image to remove the noise. Secondly, the quadratic surface
is locally fitted with the range image. The range image isexpressed by the cylindrical coordinates. Each point of theCoefficients    are determined by minimizing thefollowing equation using the range data    and Equation (7).

Laser rangefinder

Object         X4(c) Direction of camera
Figure 3. 3D-Digitizer.

range image is represented as:Ü  Ý  Þ             Ë×  Ø 
(5)




where , ,  are the distance from the center of rotation,
the angle of rotation, and the height along the rotation axis,
respectively.
The unit normal vector is given as follows:Æ  ËË ËË




 



 



(6)



    










 









 

It is important to observe the strong specular reflection
component to make calculations stable, since the specular reflection component is observed only within a limited range of angle around the reflection vector. Illumination conditions must be changed according to the shapeof measured object. To achieve this, multiple positions of alight are determined among  possible positions preparedaround the laser rangefinder as shown in Figure 4. The
light source rotates with the laser rangefinder. The numberof possible positions is 60    in the present setup
and these are arranged with 5 vertically and with 12 horizontally at the interval of 5 cm. The positions of a lightsource and a camera are calibrated by measuring a box
whose size is known. When multiple possible positions
are selected, a single light is attached at selected possible
positions in turn. Therefore, the calibration of brightness
among multiple lights is not needed. With respect to thechange of brightness of the light source by the distance between the object and the light source, it is considered as theattenuation of the light source by Equation (1).
Laser rangefinder

 



where ,  are gradient components of the range image  . This gradient is computed using the following local
surface fit:

¼   ¼
¼

¼        (10)

2.5 Selection of light source

X3 Virtual camera



 

where ,  are local coordinates in a    window. In our
approach, selected local quadratic surface fit is achieved byusing the Yokoya-Levine operator [10]. In [10], the best
window is selected among 25 windows which include thepoint    to estimate the coefficients    at   . The
best window provides the minimum fitting error in Equation (10). Then, ¼ and ¼ are computed from Equation (8),(9) and the estimated coefficients    .

(b) Motion during(a) Appearance ofmeasurement
3D-Digitizer
Laser rangefinder
X1
X2
CameraCenter of rotation



 

                        

(7)
(8)
(9)

Possible light position

Figure 4. Multiple possible light source positions.

To densely estimate non-uniform reflectance parameters independently, it is needed to observe the two reflection components separately for each pixel. However, since
the diffuse reflection component is observed over the object surface, it is impossible to observe only the specularreflection component. The specular reflection component
can be calculated by subtracting a value of a pixel which
includes only the diffuse reflection component from thatof the pixel which includes both the diffuse and specularreflection components. Therefore, it is needed to observea pixel under three different lighting conditions: One forobserving only the diffuse reflection component to acquire
one unknown parameter  based on Equation (2) and others for observing both the diffuse and specular reflection
components to acquire two unknown parameters   andby solving simultaneous equations based on Equation (3).
To observe every pixel under appropriate three different
lighting conditions, the positions of the light are selectedfrom  possible positions.
Let  be a color image which is to be obtained with apossible light position           and consists of pixels        , where  means a color intensity,
 be the number of pixels which include only the diffuse
reflection component in   , and  be the number of pixels
which include the specular reflection component stronglyin  .
First, the following items are judged for each pixel inthe object surface texture under each light position .
Measurability of the light reflectionMeasurability of only the diffuse reflectionMeasurability of the strong specular reflection

Second, the light positions  and  which satisfy            and            are selected. In the next light source position selection, the position which satisfies the same condition is selected among
the rest except for light source positions decided so far.
Then,  light positions are selected to densely estimate
reflectance parameters. The selections of positions are repeated until almost all pixels are observed once for only
the diffuse reflection component and twice for the strong
specular reflection component.
We introduce a threshold , i.e., the ratio of the measurability of both reflection components to stop the processof selecting light positions. With respect to determining thethreshold , we judge the ratio of the measurability of the
specular reflection with all possible positions. This ratio
is a limit of measuring the specular reflection component.
Using the threshold, we can terminate the light selection
process in the case that even if the number of positions oflight source is increased, the ratio of measurability of specular reflection component cannot increase more. Therefore, reflectance parameters can be efficiently estimated almost the whole object surface using a limited number of
texture images.
2.5.1 Measurability of the light reflection
In order to measure the light reflection at a specific point of
the object surface, the point on the surface must be observable from the camera position. Additionally positional relationship among the camera, the point and the light sourcemust satisfy the following conditions.

Î Æ   ¼ Ä Æ   ¼


(11)



Î Ä

Æ

where  ,  , and  are the viewing direction, the lightsource direction, and the surface normal at the -th pixel,
respectively. Note that the viewing direction  and thesurface normal  are independent of the light source position .
Even when the above equations are both satisfied, there
is a possibility that a shadow is casted on the pixel. Figure
5 shows such a case. In this case, the pixel must not be used
for estimating reflectance parameters. Whether a point iscovered by a shadow casted by light source  or not can be
judged as follows.
Let   !       be the coordinates of thepoint on the object surface,   !         be
the coordinates of the possible light position. There is abounding box which is surrounded by the maximum andminimum values on each   ! axis of position  andas shown in Figure 5. Here, we can assume that all polygons which make the object surface are small enough to becovered by the bounding box. Then, in order to examine
if a shadow is cast over the point , we have to examine if
any polygons that contain any vertices inside the boundingbox intersect with the line segment that connects and .

ÎÆLight sourcep
Shadow areaujoObject surface
Normal N

Figure 5. An illumination of detecting a selfshadow.

Therefore, for all the points inside of the bounding box, thefollowing test is performed [2].
Let  be the point which is found to be included in thebounding box and   !         be the coordinates of the point. If  lays on the plane whose normal
vector is N, the plane and the line segment which links and are given by the following equations, respectively.

 
 
"
#







" 


 



L’
Range of observing
both reflection
components


" 



"
















  "   " 









V

N
L

L’

θr

Range of observing the specularreflection component strongly

θth2
Object surface(13)

Figure 7. Observation of the specular reflection component strongly.







point . If # satisfies # and the point $ lays on
the polygon which includes , it is concluded that the line
segment of Equation (12) crosses the object.
Measurablility of the diffuse reflection only

When the -th pixel consists of only the diffuse reflection,
the reflection vector ¼ satisfies the following equation.
%



Ä



Î Ä¼   % 


Î

(14)

Ä

where %  is a threshold angle between  and ¼ .
Equation (14) implies that only the diffuse reflection component is observed if %  is greater than %  as illustrated
in Figure 6. When this condition stands and the pixel is notin a shadow, the pixel is judged to have diffuse reflection
only and is counted in   .
2.5.3

only the diffuse
reflection component





2.5.2

V Range of observing

θr



À
 Æ
Æ È
where À is the vector from the point to the surface point, È is the vector from the light position to the surfaceL

Figure 6. Observation of only the diffuse reflection component.

  " !    
(12)
 !


  "   " 

θ th1

Object surface

The intersection of the plane and the line segment, the point$ , can be obtained by solving Equation (12) about # . Now# is defined as follows.
#N

Measurability of the strong specular reflection

When -th pixel includes the specular reflection strongly,
the reflection vector ¼ satisfies the following equation.
%



Ä



Î Ä¼ %  (15)

where %  is a threshold angle between Î and Ä¼ .
Equation (15) means that both the diffuse and specular reflection components are observed if % is smaller than % 
as illustrated in Figure 7. The above condition is based on
the fact that the specular reflection is observed strongly ina limited range of a viewing angle. When this condition
stands and the pixel is not in a shadow, the pixel is judged
to have strong specular reflection and is counted in   .

2.6 Estimation of reflectance parameters
After the positions of the light source are determined,
multiple color images are taken under the different lighting conditions and then non-uniform reflectance parameters are estimated.
Let  be the set of pixels which are judged
to consist of only the diffuse reflection component
with possible light position  and consist of  pixels
       , where  means a color intensity. Let  be the set of pixels which are judged
to include the specular reflection component strongly andconsist of  pixels         , where 
means a color intensity. Note that there are some pixels which consist of neither the diffuse reflection component only, nor the strong specular reflection component in and  . Let  be such the  -th pixel. With
respect to that pixel, reflectance parameters cannot be estimated.

2.6.1 Estimation of diffuse reflectance parameter
The diffuse reflectance parameter  at the -th pixel isestimated by solving Equation (2) with the value of the -thpixel  in the image  ,  and  as follows:

Æ



 

 Æ Ä 


 



Ä

(16)

In order to get the most reliable estimation, the pixel whoseangle % is the smallest but greater than %  is selected.

2.6.2Estimation of specular reflectance and surface
roughness parameters

The specular reflectance parameter  and the surface
roughness parameter  at the -th pixel are estimated bysolving Equation (3) with the value of the specular reflection component which is extracted from the -th pixels
 and   in the images  and   ,
,
,
and
.
In
order
to
get
the
most
reliableestimation, the pixels whose angle %  is the smallest or the second
smallest and is smaller than % , are selected.
First, the diffuse reflection component is computed bysubstituting the estimated   into Equation (2). Second,
the specular reflection component is extracted by subtracting the diffuse reflection component from Equation (1). Finally, the specular reflectance and surface roughness parameters are obtained by solving the following simultaneous equations that are derived from Equation (3).

Ä

Æ

Î



             Î Ä      Æ Î  Î Ä     

 




¾







 




Æ Î 







½

 

¾

½

 

 

 

 ¾

  

¼



¼



¾



¾



where  and   are specular reflection components of the pixel obtained under different two illumination
conditions  and  as:

 
 
    where  is the diffuse reflection component obtained from  estimated in the previous section,  and by using Equation (2).
It should be noted that Equation (3) cannot be solved fora pixel at which strong specular reflection is observed lessthan twice. In such a case, the specular reflectance and surface roughness parameters are estimated by using a linearinterpolation method within a &  & window. When
the window is small against the area whose reflectance parameters are not solved, pixels are interpolated linearly byscanning the texture image horizontally.

Æ

Ä

3

Experiments

In our experiments, a measured object is a plastic dolland exibits non-uniform surface reflectance as shown in
Figure 8(a) and is assumed not to have interreflections. We
fix some parameters as %   , %   ,   and the size of the window that is used to interpolate thepixel whose reflectance parameters are not estimated is33 &  . The proposed method of selecting positions of the light source determined 12 light source positions for the object.

Figure 8(b) (d) show the estimated reflectance parameters coded as the color intensity in the cylindrical coordinates. Figure 8(b) illustrates the diffuse reflectance parameter estimated all over the object surface. Figure 8(c) is the
specular reflectance parameters. This image shows that the
specular reflectance parameter of the doll’s beak and leg
are different from the rest. Actually, the beak and legs arehighly reflective. This result is consistent with an observation of a doll in Figure 8(a). Figure 8(d) shows the surface roughness parameter with gray-scale where the largest
value is coded as white. This image means that the smaller
the value is, the smoother the object surface is. Figure 8(e)shows the ratio of pixels where specular reflectance andsurface roughness parameters are computed. The blackpart means that both parameters are not directly estimated.
Non-uniform diffuse reflectance parameter was estimatedon the whole observed object surface. Non-uniform specular reflectance and surface roughness parameters were estimated without interpolation for 83.46% of the surface.
Note that there is a part where reflectance parameters cannot be estimated. The reason is that the normal vector isa vertical upward or downward. These parts are linearlyinterpolated.
Figure 9 illustrates the measurability of both reflection
components with respect to the number of light sources. In
these graphs, the horizontal axis means the number of selected light source positions and the vertical axis means
the measurability of each reflection component. Figure9(a) and (b) show the ratio of the measurability of the diffuse and the specular reflection component, respectively.
In Figure 9(b), when the number of selected light source
position is 1, the ratio of measurability of the specular reflection component is . This is because at least 2 different light source positions are needed to determine the specular reflectance and the surface roughness parameter. With
respect to the diffuse reflectance parameter, when the number of selected light source positions is 5, the ratio of themeasurability of the diffuse reflection component is .
As for the specular reflectance and the surface roughness
parameters, even when the object is measured with all possible positions, the ratio of the measurability of the specular reflection component is . In our method, theratio of measurability of the specular reflection componentis  with automatically selected 12 light source positions. Since the ratio of measurability of both reflection
component is comparable to that using all possible lightsource positions, the experiment shows that the proposed
method is much valid.
Figure 10 shows rendered images using the estimatedparameters in Figure 8 with five different virtual lightsources. The origin of the coordinate system locates at theobject’s barycenter, the positive direction of , and ! axes

(a) Object(b) Diffuse reflectance parameter  (d) Surface roughness parameter  (c) Specular reflectance parameter  (e) Area of observed specular reflection













Ratio of measurability of
specular reflectance andsurface roughness
parameters

Ratio of measurability ofdiffuse reflectance parameter

Figure 8. A measured object and estimated reflectance parameters.

%















        

Number of selected light source positions(a) Measurability of diffuse reflectance parameter















%















        

Number of selected light source positions(b) Measurability of specular reflectance and surface
roughness parameters

Figure 9. The possibility of estimating reflectance parameters with respect to the number of light
positions.

agree with the right, upper and forward directions on theimage plane, respectively. The viewing position locates at(0.0, 0.0, 20.0) and the viewing direction is toward negative
direction of ! axis. The virtual light source rotates aroundaxis. The direction of the virtual light is captioned by the
degree of rotation in (a)(e). Let 0 degree be the front of
the object. The width, height and depth of the object are approximately 18.6 cm, 25.0 cm, and 14.8 cm, respectively.
Note that the diffuse reflectance parameter represents theobject’s texture and the highlight is observed sharply at the
doll’s leg and beak under each illumination condition.
Nevertheless, since the interreflection is not consideredin the present work, parameters for doll’s neck are not correctly estimated, where the observed intensity consists ofsecondary reflection from some object part. Moreover, because some part in the range image is blank with measuring
by the laser rangefinder, the object is not rendered well.

4

Conclusions

In this paper, we have proposed a new method ofdensely estimating non-uniform reflectance properties foralmost the whole object surface by using the laser
rangefinder. In our approach, multiple light source positions around the laser rangefinder are automatically selected, so that two reflection components are observed
densely. The experiments have shown that the proposed
method is useful for estimating the reflectance parametersof objects which exhibit non-uniform surface reflectance.
However since modeling a light includes some errors and(a) light 1 (-45.0 °)(b) light 2 (-22.5 °)(c) light 3 (0.0 °)(d) light 4 (22.5 °)(e) light 5 (45.0 °)Figure 10. Rendering of object with virtual light sources.
interreflections are not considered, reflectance parameters
are not estimated stably in some part. In the future, we will
improve the method so that a light is modeled accurately.
We will also investigate a method to consider interreflections all over the object surface.

References[1] R. Baribeau, M. Rioux, and G. Godin. Color reflectancemodeling using a polychromatic laser sensor. IEEE Trans.
on Pattern Anal. Mach. Intell., 14(2):263–269, 1992.
[2] J. Foley, A. van Dam, S. Feiner, and J. Hughes. Computer
Graphics Principles and Practice. Addison-Wesley Publishing Company, 2nd edition, 1993.
[3] K. Ikeuchi and K. Sato. Determining reflectance properties of an object using range and brightness images. IEEE
Trans. on Pattern Anal. Mach. Intell., 13(11):1139–1153,
1991.
[4] G. Kay and T. Caelli. Inverting an illumination model fromrange and intensity maps. CVGIP: Image Understanding,
59:183–201, 1994.
[5] S. Lin and S. W. Lee. Estimation of diffuse and specularappearance. Proc. Int. Conf. on Computer Vision, 2:855–
860, 1999.
[6] S. Lin and S. W. Lee. A representation of specular appearance. Proc. Int. Conf. on Computer Vision, 2:849–854,
1999.
[7] J. Lu and J. Little. Reflectance function estimation andshape recovery from image sequence of a rotating object.
Proc. Int. Conf. on Computer Vision, pages 80–86, June
1995.
[8] Y. Sato, M. D. Wheeler, and K. Ikeuchi. Object shape andreflectance modeling from observation. Proc. SIGGRAPH’97, pages 379–387, 1997.
[9] K. E. Torrance and E. M. Sparrow. Theory for off-specularreflection from roughened surfaces. Journal of Optical Society of America, pages 1105–1114, 1967.
[10] N. Yokoya and M. D. Levine. Range image segmentation based on differential geometry: A hybrid approach.

IEEE Trans. Pattern Anal. Mach. Intell., 11(6):643–649,
June 1989.