Dense Estimation of Surface Reflectance Propertiesof Objects with Interreflections

Abstract
In augmented virtuality which virtualizes real objects toconstruct a mixed reality environment, it is important to estimate object surface reflectance properties to render objects under arbitrary illumination conditions. The authors
developed a method to estimate reflectance properties ofobject surfaces densely. However, it was difficult to estimate surface reflectance properties faithfully for objects
with interreflections. This paper describes a new method ofdensely estimating non-uniform surface reflectance properties of real objects constructed of convex and concave surfaces with interreflections. We use registered range and surface color texture images obtained by a laser rangefinder.
The proposed method first determines positions of light totake color images for discriminating diffuse and specularreflection components of surface reflection. Then, surface
reflectance properties are estimated based on radiosity. Experiments show the usefulness of the proposed method.

1

Introduction

In constructing a mixed reality (MR) environment, it isrequired to obtain surface reflectance properties of virtualized real objects. For this purpose, we have conducted a
research to estimate surface reflectance properties denselyfrom range and surface texture images [3].
Object surface reflectance properties consist of diffuse
and specular reflection components. The diffuse reflection
component is easily observed due to its nature of reflectionin omni-direction. On the other hand, it is difficult to observe the specular reflection component due to its nature ofreflection within a fixed angle with respect to a viewing position, light source and object surface normal vector. Moreover, even if the specular reflection component is observed,
this component cannot be estimated stably due to small values [1, 2, 4]. Our resent research has solved this problem
by selecting optimum light positions to observe both components [3]. However, there still exists a problem such thatinterreflections are not considered.

In this paper, we take notice of the radiosity rendering
method which can represent interreflections based on thetransmission of radiosity (heat energy of light). Yu et al.
[7] have estimated surface reflectance properties of a roomfrom color and range data considering interreflections basedon the radiosity method. They, however, assume that almost
all objects have the same reflectance properties, therefore,
their algorithm cannot be applied to an object which has
non-uniform surface reflectance properties.
We propose a new method for estimating non-uniform
reflectance properties of objects with interreflections torepresent virtualized objects photo-realistically. The proposed method can observe the specular reflection component densely considering interreflections based on radiosity.

2

Estimation of reflectance parameters fromrange and color images

Figure 1 shows a flow diagram of estimating surface reflectance properties. Our process consists of three parts,
which are a measurement of an object (A,C), a selectionof light source (B), and an estimation of reflectance parameters (D).

2.1

Measurement of object

We use a laser rangefinder (Cyberware 3030RGB) withknown positions of point light sources and a camera for acquiring surface color images, as illustrated in Figure 2(a).
This system can obtain registered range and surface color
texture images at the same time by rotating the rangefinder
and the camera around an object, so that there is no registration error, even when an object is measured many times.
Figure 2(b) shows the illustration viewed from the top ofthe device. A camera is located at  and a texture imageis acquired through mirrors which are located at  and. We assume that the camera is virtually located at and the camera looks toward the center of rotation.
Generally, the noise and quantization error are includedin the range image acquired from the laser rangefinder.
There is also a problem that it is difficulat to calculateLaser rangefinder

A. Measurement
Range data
Possible light position 1
Ratio of observing eachreflection component. . . .

Possible light position M
Ratio of observing eachreflection component

B. Selection of light source position

Possible light position

Figure 3. Multiple possible light source positions.

C. Measurement
Selected light positoin 1
Surface texture image. . . .

Selected light position m
Surface texture image 

D. Estimation of diffuse reflectance parameter based on radiosity
Estimation of specular and surface roughness parameters

Figure 1. Flow diagram of estimating surface
reflectance properties.

Center of rotation

Laser rangefinder
X1
X2
CameraX3 Virtual camera
X4(a) Appearance(b) Direction of cameraof 3D-Digitizer
Figure 2. 3D-Digitizer.

Selection of positions of light source

In the present experimental setup, multiple positionsof a light are determined among 60 possible positions
prepared around the laser rangefinder and these are twodimensionally arranged at the interval of 5 cm as shownin Figure 3. The positions of a camera and a light source
are calibrated in advance. When optimum light positions
are selected, a single light is attached at the selected positions in turn. Therefore, the calibration of brightness among
multiple lights is not needed.
Here, we employ the Torrance-Sparrow model [5] to represent object reflectance properties physically for selecting
light positions. The Torrance-Sparrow model is given as:









      

(1)where represents an observed intensity,
is an attenuation coefficient concerning the distance between a point
light source and an object surface point,represents the

the surface normal accurately around the discontinuity inthe range image. Therefore, we employ an adaptive local
quadratic surface fitting [6] as a preprocessing.

2.2

strength of a light souce.  ,  and are the diffuse reflectance, the specular reflectance and the surface roughness parameters, respectively.  is an angle between a lightsource vector and a surface normal vector,  is an angle between a viewing vector and a surface normal vector, and 
is an angle between a viewing vector and a reflection vector. Note that the reflection vector is a vector to which alight vector is mirrored against a normal vector.
To densely estimate non-uniform reflectance parameters
independently, it is required to observe each pixel under
three different lighting conditions: One for observing only
the diffuse reflection component to determine one unknown
parameter  and others for observing both the diffuse andspecular reflection components to acquire two unknown parameters  and . Therefore, the selection of position ofa light source is repeated until almost all pixels satisfies
three different lighting conditions. Then, a certain numberof light positions, say , are selected to densely observeboth reflection components.

2.3

Estimation of reflectance parameters

A texture image is obtained with a selected light position       and consists of pixels  ½     ,
where  means a color intensity. Each pixel is classified
into three types  ,  and   .  means a pixel
containing only the diffuse reflection component and means a pixel containing strong specular reflection component.   means a pixel classified neither applies to 
nor to  and is not used to estimate reflectance parameters.







2.3.1 Estimation of diffuse reflectance parameterbased on radiosity
We assume that interreflections do not have an influence
from both reflection components but only from the diffuse
reflection component. Therefore, a pixel which is categorized in  is used in the following method. In this paper,to consider interreflections, we employ the radiosity equation given as:   (2)Laser rangefindertion component extracted from two images taken under two
different illumination conditions and  estimated above.
See [3] for detail. Note that  estimated above should bescaled to Equation (1) before computing the specular andsurface roughness parameters.

p(t)Objects u tp(s)3Experiments

Figure 4. Calculation of radiosity.

 and   
 represent the radiosity ofwhere-th and -th patch, respectively. Note that is the numberof object patches.
 is a form factor between the patch
and the patch .
is an enviroment term which has
an influence on the patch . In this paper, we assume that , because the object is measured in a dark room. represents the diffuse reflectance parameter on the patch .
In the proposed method, a form factor  is known because the object shape is measured by the laser rangefinder.
Since the range and texture images are registered at eachpixel, the radiosityof the patch is calculated basedon the value of the pixel which corresponds to the patch .
Then, the diffuse reflectance parameter  is estimated byEquation (). Finally, the diffuse reflectance parameter ata each surface point is estimated by calculating the averageamong neighbouring patches which share the point.
Here, let   be the position of a light source when apixel is measured as shown in Figure 4,  be the valueof the pixel . The radiosityof the patch which includes pixels and is calculated by the following method.
Each patch consists of four points andis represented
as the sum of the values of the pixels which correspond tothese points. Because a light source attached with the laser
rangefinder moves during measurement, it is necessary tocalculate  with the light position  .  is interpolated
linearly by the following equation since  and  are categorized in  .

              (3)where  and  are normal vectors at pixels  and .
´µ and ´µ are light vectors with the light position
 at pixels  and . Note that in the case of   , should be 0 to consider the measurability of light reflection.

2.3.2 Estimation of specular reflectance and surface roughness parameters based on TorranceSparrow model

The specular reflectance parameter  and the surface
roughness parameter  at the -th pixel which is categorized in  are estimated by solving a simultaneous equation of Equation (1) with the values of the specular reflec-Measured objects in our experiments are shown in Figure 5(a)(c). Object A exhibits only the diffuse reflection.
On the other hand, object B and C exhibit non-uniform
reflectance properties and complicated shape. For exsample, in object B, the leg’s reflectance properties are different
from the body’s reflectance properties.
In a preliminary experiment, we separate the object A
into two parts region A and region B. The same white paper with a uniform reflectance is pasted up on both regions.
The proposed method is compared with the conventional
method [3]. Results are shown in Figure 6. Each graph represents RGB channels of the diffuse reflectance parameter
estimated by both methods. A horizontal axis means theposition of the pixel in the vertical direction of the object
and a vertical axis means the average of diffuse reflectanceparameters in the horizontal direction of the object. In theconventional method, the value of the diffuse reflectance parameter is large around the boundary between regions A andB because the conventional method does not consider interreflections. On the other hand, in the proposed method, thevalue estimated is more stable than the value estimated by
the conventional method. This clearly shows the usefulnessof the proposed method.
In the next experiment, we measure objects B and C. The
measurability of both reflection components and the number of selected light sources are shown in Figure 5(d). The
number in the bracket means the number of selected positions of a light source required to estimate the diffuse reflectance parameter. From these results, it is observed that
non-uniform surface reflectance properties are estimatedover almost the whole surface considering interreflections
with a limited number of light positions.
Figure 7 shows the rendering of virtualized objects B andC under virtual lighting conditions. A virtual light source
moves on a circle whose center is on the vertical axis ofthe object. The position of the virtual light source is represented as the angle from the front of the object. The
Torrance-Sparrow model is employed to render the object.
Each object is rendered suitably at the part which contains
interreflections. However, at the legs of object B and thelower part of object C, interreflections due to the specularreflection component cause some errors. It is necessary toeliminate such an influence.

Region AObject
LightDiffuse
Specular

Region BB12(5)100.0%
83.5%

C
14(4)
100.0%
91.0%

Conventional method
Proposed method

Region A

Region B







       
Conventional method

Proposed method

Region BRegion A       T he position of pixel in virtical direct ion

T he posit ion of pixel in virtical direct ion

The average of the diffuse
reflectance parameterThe average of the diffuse
reflectance parameter

The average of the diffuse
reflectance parameter(d) measurability(b) Object B(a) Object A(c) Object C
Figure 5. Three objects used in experiment and measurability of two reflection components.
Conventional method
Proposed methodRegion B       T he posit ion of pixel in virt ical direction(b) G channel(a) R channelRegion A(c) B channel
Figure 6. A comparison with previous work in a preliminary experiment.

 4  (b) Object C(a) Object B
Figure 7. Rendering of virtualized objects with Torrance-Sparrow model.

Conclusions

In this paper, we have proposed a new method of densely
estimating non-uniform reflectance properties of real objects for virtualizing the objects. In our approach, we haveconsidered interreflections by estimating surface reflectanceproperties based on radiosity. The experiments have shownthat the proposed method is useful for estimating the reflectance parameters of objects which exhibit non-uniform
surface reflectance.

References[1] S. Lin and S. W. Lee. Estimation of diffuse and specular appearance. Proc. Int. Conf. on Computer Vision, 2:855–860,1999.

 [2] S. Lin and S. W. Lee. A representation of specular appearance. Proc. Int. Conf. on Computer Vision, 2:849–854, 1999.
[3] T. Machida, H. Takemura, and N. Yokoya. Dense estimation of surface reflectance parameters by selecting optimum
illumination conditions. Proc. Vision Interface 2001, pages
244–251, June 2001.
[4] Y. Sato, M. D. Wheeler, and K. Ikeuchi. Object shape and reflectance modeling from observation. Proc. SIGGRAPH ’97,pages 379–387, 1997.
[5] K. E. Torrance and E. M. Sparrow. Theory for off-specular reflection from roughened surfaces. Journal of Optical Societyof America, pages 1105–1114, 1967.
[6] N. Yokoya and M. D. Levine. Range image segmentation
based on differential geometry: A hybrid approach. IEEE
Trans. Pattern Anal. Mach. Intell., 11(6):643–649, June 1989.
[7] Y. Yu, P. E. Debevec, J. Malik, and T. Hawkins. Inverse global
illumination: Recovering reflectance models of real scenes
from photographs. Proc. SIGGRAPH ’99, pages 215–227,
August 1999.