Dense Estimation of Surface Reflectance Properties
Based on Inverse Global Illumination Rendering
Takashi Machida , Haruo Takemura and Naokazu Yokoya
Cybermedia Center, Osaka University
Graduate School of Information Science, Nara Institute of Science and Technology
E-mail: machida, takemura@ime.cmc.osaka-u.ac.jp, yokoya@is.aist-nara.ac.jp

Abstract
In augmented virtuality, estimating object surface reflectance properties is important when rendering objects
under arbitrary illumination conditions. However, faithfully
estimating surface reflectance properties is difficult for objects having interreflections. The present paper describes
a new method for densely estimating the non-uniform surface reflectance properties of real objects constructed ofconvex and concave surfaces having diffuse and specularinterreflections. The registered range and surface color texture images were obtained using a laser rangefinder. In theproposed method, the light positions are first determinedin order to take color images, which are then used to discriminate diffuse and specular reflection components of surface reflection. Surface reflectance parameters are then estimated based on an inverse global illumination rendering.
Experiments were conducted to reveal the usefulness of theproposed method.

1Introduction

In constructing a mixed reality (MR) environment, an inverse global illumination rendering is conducted in order toobtain the surface reflectance properties of virtualized real
objects [2, 5]. For this purpose, we have studied the dense
estimation of surface reflectance properties from range andsurface texture images with diffuse interreflections [7]. In
our previous research, an appropriate number of light positions are selected so that both diffuse and specular reflection components can be observed. Reflectance parameters
can be then estimated. However, there remains a problem
whereby specular interreflections cause errors.
Yu et al. [10] have estimated the surface reflectanceproperties of a room from color and geometry data considering both diffuse and specular interreflections based on
inverse global illumination rendering. Boivin et al. [1]have also attempted to estimate surface reflectance properties considering diffuse interreflections. However, both ofthese methods assume that the surface of interest has uniform reflectance properties and therefore their algorithms
cannot be applied to an non-uniform surface reflectance object.
In the present paper, we consider photon mapping [3],which is a global illumination rendering method. Photon
mapping can represent diffuse and specular interreflections
based on the emission of a photon from a light source. We
propose herein a new method, using photon mapping, forestimating the non-uniform reflectance properties of objects
with both diffuse and specular interreflections. In addition,
the proposed method can densely estimate reflectance parameters by densely observing both reflection components.

2

Estimation of reflectance parameters fromrange and color images

Figure 1 shows a flow diagram of the estimation of surface reflectance properties. The proposed process consistsof five procedures: measurement of the object (A,C), selection of a light source (B), initial estimation of reflectanceparameters (D), and reflectance estimation via photon mapping (E).

2.1

Measurement and selection of light source positions

A laser rangefinder (Cyberware 3030RGB) along withknown positions of a point light source and a camera areused to acquire surface color images, as illustrated in Figure 2(a). This system can obtain registered range and surface color texture images simultaneously by rotating therangefinder and the camera around the object of interest.
In the present experimental setup, multiple positionsof a light source are determined among 60 possible positions around the laser rangefinder, and these are two-reflection component.   pixels are pixels that cannot beclassified as neither  or  .

A. Measurement
Range data
B. Selection of optimum positions of light source
Selected light position 1

2.2… Selected light position mEstimation of reflectance parameters using
photon mappingC. Measurement
Surface texture image 1… Surface texture image m

D. Initial estimation of diffuse reflectance, specular reflectanceand surface roughness parameters by our previous method
Range data, texture images and initial reflectance parameters
E. Estimation of reflectance parameters based on photonmap

Figure 1. Flow diagram of estimating surface
reflectance properties
Laser rangefinder

2.2.1 Photon mapping
In the photon mapping rendering method [3], an outgoing radiance from a surface point is calculated inorder to decide a surface color. The following equations
constitute the rendering equations in the photon mapping
method.

Possible light position(a) Appearanceof 3D-Digitizer(b) Multiple possible light source
positionsFigure 2. Experimental setup

dimensionally arranged at intervals of 5 cm, as shown in
Figure 2(b). The positions of the camera and light source
are calibrated in advance.
Here, we employ the Torrance-Sparrow model [8] torepresent object reflectance properties physically. The
Torrance-Sparrow model is given as:      

As an initial estimation, the reflectance parameters areobtained using our previous method [6] in Figure 1(D). The
diffuse reflectance parameter is estimated considering diffuse interreflections. The specular reflectance and the surface roughness parameters are estimated with no consideration of specular interreflections. Here, let ¼ , ¼ and¼ be the reflectance parameters obtained in this process.
These parameters are used as initial parameters for the next
process (E) in Figure 1.

(1)

where denotes the surface point,  represents an observedintensity,  is an attenuation coefficient concerning the distance between a point light source and an object surface
point, and  represents the strength of a light source.   , and  are the diffuse reflectance, specular reflectanceand surface roughness parameters, respectively.  is theangle between a light source vector and a surface normalvector,  is the angle between a viewing vector and a surface normal vector, and  is the angle between a viewing vector and a reflection vector. In order to densely estimate non-uniform reflectance parameters independently,
optimum light positions for observing both reflection components at each surface point must be determined [6].
A texture image is obtained along with a selected lightposition          and consists ofpixels
½      , where  is the color intensity of a surface
point . Each pixel is classified as one of three types  , or   .  pixels contain only the diffuse reflection component and  pixels contain a strong specular  


 






 


¼                       Ü                   

¼

¼

¼(2)
¼(3)

where ,  ,and are emitted radiance, reflected radiance, incoming radiance, and a BRDF (i.e. the TorranceSparrow model), respectively. Note that the outgoing radiance is equivalent to the reflected radiance  due to theassumption that the object has no emission.
Equations (2) and (3) are theoretical models. The color
 at a surface point is represented by the following equation obtained from Eq. (1) [9].

      (4)where  is the incoming radiance.        denotes
the specular term in Eq. (1), and other parameters are thesame as in Eq. (1).  is decided by counting the numberof photons which arrive at the point . Photons are traced
using a Monte Carlo ray tracing method [4]. In this case,the photon is reflected or absorbed depending on the reflectance parameters, and only the photons that are reflected
are traced iteratively.

Range data, texture images

Initial reflectance parameters( Pdx0 Psx0 σ x0 )(a). Estimation of diffuse reflectance parameter
Rendering based on Photonmap
Estimation of diffuse reflectance parameter Pdx
using Equation (5)nUpdate of reflectance parameter Pdx
nReflectance parameters( Pdx Psxn −1 σ xn −1 )(b). Estimation of specular reflectance and surface roughness parameter
Rendering based on Photonmap
Estimation of specular reflectance parameter Psx andsurface roughness parameter σ x using Equation (5)nUpdate of reflectance parameters Psx σ xnn

Reflectance parameters( Pdx Psxn σ xn )Figure 3. Detail of reflectance estimation process based on photonmap
2.2.2 Iterative estimation of reflectance parameters
Figure 3 details the iterative reflectance estimation process shown in Figure 1 (E). Our approach uses the photon mapping method. The reflectance parameter estimation
method based on photon mapping is separated into two processes (a) and (b), as shown in Figure 3. These processes
are conducted iteratively. In each process, we minimize thefollowing equation derived from Eq. (4) at each pixel in thetexture image.

 

    (5)where  is the measured radiance (color intensity) and is the irradiance computed from Eq. (4) at the surface point
for light source position  . Here,  denotes the number ofsample points.
In process (a), the diffuse reflectance parameter  isestimated using a  pixel. ¼ , ¼ and ¼ are used tocompute  only at the first iteration. Here, the specularreflection term in Eq. (4),         , is set to
be 0 because the specular reflection can not be observed.
In process (b), the specular reflectance  and the surface roughness  parameters are estimated using only pixels categorized as  or   . Again, ¼ and ¼ areused to compute  only at the first iteration.   estimatedabove is used in Eq. (4). When  and  are estimated,the value of each reflectance parameter is updated and processes (a) and (b) are iterated until        falls
below a threshold .

3Experiments

A conventional PC (Pentium 4 3.06 GHz, memory: 2GB) is used in the following experiments. The numberof photons is 2 million and our algorithm requires approximately 4 hours to estimate the reflectance parameters ofeach object.
As a preliminary experiment, we used Object A (Figure4(a)), and compared the proposed method to the conventional method I (which does not consider interreflections)[6] and II (which considers only diffuse interreflections) [7].
Object A consists of two plates (region  and  ) that are
positioned orthogonally to one another and have the same
glossy reflectance properties. The results of the comparison are shown in Figure 5. Each graph represents the RGB
channels of the diffuse reflectance parameter estimated by
one of the three methods. The horizontal axis represents theposition of the pixel along the vertical direction of the object, and the vertical axis represents the average of the diffuse reflectance parameters along the horizontal direction ofthe object. In both conventional methods I and II, the diffuse
reflectance parameter is large near the boundary between
regions  and  due to the effect of interreflections. In contrast, the parameter estimated using the proposed method,is more stable.
In the next experiment, we use Objects B and C shown in
Figures 4(b) and (c). These objects have non-uniform diffuse and specular reflectance properties. Figure 4(d) shows
the measurability of both reflection components and thenumber of selected light sources. The number of selected
light sources for estimating the diffuse reflectance parameter is given in brackets. Figure 6 shows real images anddifferences between real and synthetic cylindrical images(rendered by photon mapping) for the conventional method
II and the proposed method. The light position is above
the rangefinder. The synthetic images were rendered using the reflectance parameters estimated under the same illumination conditions as those for the real images. Notethat linear interpolation is conducted when the specular reflectance and the surface roughness parameters can not be
estimated due to their small specular reflection. In the conventional method, the error due to the influence of specularinterreflections is confirmed. In particular, Objects B andC have large errors at inequalities (i.e. at the cat’s leg andneck and at the pig’s nose). The proposed method does nothave such a disadvantage. In addition, Table 1 shows theaverage and variance of differences between real and synthetic images. The proposed method provides an image thatis much closer to the real image, compared to the conventional method.

4Conclusions

In the present paper, we have proposed a new method for
densely estimating the non-uniform reflectance propertiesof real objects based on the global rendering technique. In
future research, we intend to reduce the computational cost

Object
LightDiffuse
SpecularRegion αRegion βB
10(3)100.0%
89.2%

C
9(3)
100.0%
94.7%

Conventional method I140
Conventional method II130
120110

Proposed method

100
1214161

Region αRegion β81 101 121 141 161 181 201 221

T he position of pixel in virt ical direction

150

Conventional method I140

Conventional method II130120110

Proposed method

100
12141 61Region αRegion β81 101 121 141 161 181 201 221

T he position of pixel in virt ical direction

The average of the diffuse
reflectance parameter

150

The average of the diffuse
reflectance parameter

The average of the diffuse
reflectance parameter(d) Measurability(c) Object C(b) Object B(a) Object A
Figure 4. Three objects used in experiments and measurability of two reflection components
150

Conventional method I140
Conventional method II130
120
110

Proposed method

100
1214161Region αRegion β81 101 121 141 161 181 201 221

T he position of pixel in virt ical direction(b) G channel(a) R channel(c) B channel
Figure 5. Comparison with previous research conducted in a preliminary experiment(a) Real image(b) Difference between real and synthetic (c) Difference between real and synthetic
images in the conventional method II
images in the proposed method(d) Real image(e) Difference between real and synthetic (f) Difference between real and synthetic
images in the conventional method II
images in the proposed method

Figure 6. Differences between real and synthetic cylindrical images for Objects B and CTable 1. Average and variance of differences
between real and synthetic images
Object B Object C
Average Conventional method II8.7
13.3
Proposed method
0.51
0.92
Variance Conventional method II
493.3
375.2
Proposed method
3.2
9.8
by using a GPU and to synthesize a virtualized object into areal world in real-time.

References[1] S. Boivin and A. Gagalowicz. Image-based rendering of diffuse, specular and glossy surfaces from a single image. Proc.
SIGGRAPH ’01, pages 107–116, August 2001.
[2] P. Debevec. Rendering synthetic objects into real scenes:
Bridging traditional and image-based graphics with global
illumination and high dynamic range photography. Proc.
SIGGRAPH ’98, pages 189–198, 1998.
[3] H. W. Jensen. Realistic Image Synthesis Using Photon Mapping. A K Peters, Ltd, 1st edition, 2001.

[4] J. T. Kajiya. The rendering equation. Proc. SIGGRAPH ’86,pages 143–150, August 1986.
[5] C. Loscos, G. Drettakis, and L. Robert. Interactive virtual relighting of real scenes. IEEE Trans. Visualization and Computer Graphics, 6(4):289–305, December 2000.
[6] T. Machida, H. Takemura, and N. Yokoya. Dense estimation of surface reflectance properties for merging virtualized
objects into real images. Proc. ACCV 2002, pages 688–693,January 2002.
[7] T. Machida, N. Yokoya, and H. Takemura. Surface reflectance modeling of real objects with interrefletions. Proc.
9th Int. Conf. on Computer Vision, pages 170–177, October
2003.
[8] K. E. Torrance and E. M. Sparrow. Theory for off-specularreflection from roughened surfaces. Journal of Optical Society of America, 57(9):1105–1114, 1967.
[9] G. J. Ward. Measuring and modeling anisotropic reflection.
Proc. SIGGRAPH ’92, pages 265–272, July 1992.
[10] Y. Yu, P. E. Debevec, J. Malik, and T. Hawkins. Inverse
global illumination: Recovering reflectance models of realscenes from photographs. Proc. SIGGRAPH ’99, pages
215–227, August 1999.