Using Extended Light Sources for Modeling Object Appearance
under Varying Illumination
Imari Sato
National Institute of Informatics
imarik@nii.ac.jp

Takahiro Okabe Yoichi Sato Katsushi Ikeuchi
The University of Tokyo{takahiro,ysato,ki}@iis.u-tokyo.ac.jp

Abstract
In this study, we demonstrate the effectiveness of using extended light sources for modeling the appearance of an object for varying illumination. Extended light sources havea radiance distribution that is similar to that of the Gaussian function and have the potential of functioning as a lowpass filter when the appearance of an object is sampled underthem. This enables us to obtain a set of basis images of an
object for variable illumination from input images of the object taken under those light sources without suffering aliasing
caused by insufficient sampling of its appearance. Furthermore, extended light sources are useful in terms of reducing
high contrast in image intensities due to specular and diffuse
reflection components. This helps us observe both specularand diffuse reflection components of an object in the same image taken with a single shutter speed. We have tested our proposed approach based on extended light sources with objectsof complex appearance that are generally difficult to modelusing image-based modeling techniques.

1. Introduction
The appearance of an object greatly changes depending on
its surface reflectance properties and the lighting conditionsof the location where the object is placed. For the task ofobject recognition and image synthesis, it is thus important tobe able to predict the variation of an object appearance undervarying illumination conditions. In this paper we investigate
the effectiveness of using extended light sources for modelingthe appearance of an object under varying illumination.
While there may seem to be a large variety of possible
appearances for a given object, it has been demonstrated inprevious research that the appearance changes of an object
for varying illumination can be represented with a linear subspace spanned by a set of basis images of the object. For
instance, in the case of a convex Lambertian object, its appearance seen under distant illumination without attachedand cast shadows can be described with a 3-D linear subspace spanned from three input images of the object taken
under linearly independent lighting conditions [10, 20, 25].
Even taking into account attached shadows, most of the image variation of a human face or other object under vary-Real Image

Synthesized Image

Figure 1. Synthesized appearance modeled byextended light sources.
ing illumination was shown to be adequately represented bya low-dimensional linear subspace slightly higher than 3-D[6, 3, 24]. A similar observation was utilized for object recognition in [4, 5].
A set of basis images spanning such a linear subspace areoften provided by applying principal-component analysis tothe input images of an object taken under different lighting
conditions.1 Since little is known about how to sample theappearance of an object in order to obtain its basis images
correctly, a large number of input images taken by moving apoint light source along a sphere surrounding the object are
generally provided. Then the question is whether the use of apoint light source is suitable for capturing the variation of theobject’s appearance. As one solution to this problem, Shimand Chen introduced a statistical approach that used training
scenes with different shapes and surface materials for finding
the most efficient lighting patterns to model the appearancechanges of an object [19].
Recent investigations in frequency-space analysis of reflection have shown that the appearance of an object undervarying complex illumination conditions can be well repre1 In the context of texture synthesis, Malzbender et al. took an alternative
approach of modeling the luminance dependencies of a texture on lighting
directions with biquadratic polynomials [8].

Proceedings of the IEEE International Conference on Computer Vision (ICCV 2005), pp. 325-332, October 2005

sented with a linear subspace spanned by basis images of theobject, called harmonic images, each of which corresponds
to an image of the object illuminated under harmonic lights
whose distributions are specified in terms of spherical harmonics [14, 15, 1].2 Hence if harmonic lights can be physically constructed in a real setting, harmonic images of a real
object can be obtained simply as images of the object seen
under these light sources. However, harmonic lights are complex diffuse light sources comprising both negative and positive radiance and are thus difficult to physically construct ina real setting. Therefore, most of the previously proposed
techniques synthetically compute harmonic images from theknowledge of an object 3-D shape and reflectance properties.
Lee et al. introduced the interesting concept of object
classes for determining a configuration of nine light source
directions. Input images taken under those particular lightsource directions approximate a 9-D subspace spanned byharmonic images [7]. This method has the advantage thatit requires only synthetically provided harmonic images forone object in each class in order to determine the nine lighting directions.
Recently, we introduced a method incorporating the sampling theorem of spherical harmonics for determining a set oflighting directions to efficiently sample the appearance of an
object using point light sources [17]. The sampling theorem
specifies how to substitute a set of realizable light sources forharmonic lights, and harmonic images of the object are computed as weighted sums of the sampled appearance of the object under those point light sources. While this method does
not require knowledge about the 3-D shape and reflectanceproperties of an object, information about the highest frequency that the appearance of an object contains must be provided in order to determine the number of point light sources
required. How to provide such information for a given object
remains unsolved.
The sampling theorem states that the higher the frequency
content of an object’s appearance, the more input images are
required to obtain a correct set of basis images. The numberof input images required may become extremely large in thecase of highly specular surfaces containing a large quantity ofhigh frequency components in their reflection. In this case,insufficient sampling of an object appearance will result inaliasing in the basis images, and this will lead to undesirable
artifacts in the synthesized appearance.
Since the number of input images provided for modeling an object’s appearance is usually limited, an anti-aliasing
framework for obtaining a set of correct basis images from aninsufficient number of object input images is needed. However, this aliasing problem has not been carefully consideredin previous studies.
In this study, we propose a novel approach for sampling
the appearance of an object under appropriately provided Extended Light Sources (ELS). The use of ELS for modeling the
2 Harmonic images have been also used for the purpose of efficient rendering of an object under complex illumination [16, 21].

shape and reflectance of an object was originally introducedin [11]. We extend their analysis further in the angular frequency domain so that the harmonic images of an object ofarbitrary surface materials can be obtained without sufferingfrom aliasing caused by insufficient sampling of its appearance.
The use of ELS has the following advantages. ELS havea radiance distribution that is similar to that of the Gaussianfunction, and this enables extended sources to function as a
low-pass filter when the appearance of an object is sampled
under them. From this, we are able to obtain a set of basis images of an object for varying illumination without sufferingfrom aliasing caused by insufficient sampling of its appearance. In addition, ELS can reduce high contrast in image
intensities due to specular and diffuse reflection components.
This helps avoid saturation so that we are able to observeboth specular and diffuse reflection components in the sameimage taken with a single shutter speed. We have tested our
proposed approach based on extended sources with objectsof complex appearance that are generally difficult to modelusing image-based modeling techniques.
The rest of this paper is organized as follows. We briefly
review harmonics image representation based on sphericalharmonics in Section 2, and discuss the difficulties of obtaining harmonic images of real objects in Section 3. We provide the frequency analysis of ELS in Section 4 and present
a method for modeling object’s appearance by using ELS in
Section 5. Finally, we show experimental results of the proposed method applied to synthetic and real data in Section 6
and present concluding remarks in Section 7.

2 Object Appearance under Variable Illumination
Previous studies have shown that appearance of an object
under varying illumination can be well represented with a linear subspace spanned by basis images [14, 15, 1], called harmonic images. In this section, we outline the harmonic image
representation that is based on spherical harmonics.
Spherical harmonics define an orthonormal basis over theunit sphere, whose unit vector is represented by the polar coordinate system θ, (0 ≤ θ ≤ π) in elevation and φ, (0 ≤ φ <2π) in azimuth. Then a function f (θ, φ) defined over the unit
sphere can be expanded as a linear combination of sphericalharmonicsf (θ, φ) =

∞ 
l


Flm Ylm (θ, φ),(1)

l=0 m=−land Flm are coefficients in its spherical harmonic expansion
computed as 2π  πmf (θ, φ)Ylm (θ, φ) sinθdθdφ.
(2)
Fl =
0

0

Ylm (θ, φ), (l≥ 0, −l ≤ m ≤ l) are spherical harmonics
defined as Ylm (θ, φ) = Nlm Plm (cosθ)eImφ , and Nlm are theProceedings of the IEEE International Conference on Computer Vision (ICCV 2005), pp. 325-332, October 2005

normalized constants, and Plm (·) are the associated Legendre
functions of degree l and order m.

2.1 Representing Object Appearance based on
Spherical Harmonics
A bidirectional reflectance distribution function (BRDF)
characterizes the reflectance property of an object: (θi , φi )and (θo , φo ) are incident and reflection directions with respect to the surface normal of the object surface whose local
coordinate is denoted by using  .
Representing light source distribution L(θ, φ) by the
global coordinate defined on the unit sphere, the brightnessof the object surface is computed asI=

Z π Z 2π00L(θ, φ)ρ(M (θ, φ), θo , φo ) cos θ sin θdθdφ,(3)where M (·) represents a rotation operator that rotates (θ, φ)into the local coordinate.
Considering the appearance changes of an object undervariable illumination seen from a fixed viewpoint (θo , φo ),ρ(M (θ, φ), θo , φo ) cos θ can be represented by using a global
coordinate R(θ, φ). In other words, R(θ, φ) represents how
much of the incident light from the direction(θ, φ) is reflectedon the object surface toward the fixed viewpoint (θo , φo ). In
the following, R(θ, φ) is referred to as the reflection kernel.
Since both the light source distribution and the reflection
kernel are functions defined on the unit sphere, coefficientsmLml and Rl in their spherical harmonic expansion can becomputed from (2). From the orthonormality of sphericalharmonics, the surface brightness I in (3) is computed inmterms of Lml and Rl as
I=

∞ 
l


m
Lm
l Rl .

(4)

l=0 m=−lImages containing Rlm for all pixels, that is, for all corresponding points on the object surface, are called harmonic
images [1].

3 Obtaining Harmonic Images via Point Light
Sources
While harmonic images are efficient for representing theappearance of objects under varying illumination, it is not soeasy to provide harmonic images for various kinds of real
objects. Therefore, most of the previously proposed techniques for object recognition and image synthesis provide areflection kernel of an object from the knowledge of its 3D
shape and reflectance properties and synthetically computethe coefficients Rlm from (2) for all pixels in harmonic images. However, depending on the surface materials of an object, it can be difficult to provide such information.
Recently, we introduced a method to sample the reflection kernel R(θ, φ) of an object as an observed brightnessof the surface point when the object is illuminated under alight source from the direction (θ, φ) [17].3 In particular, thismethod made use of the sampling theorem of spherical harmonics for determining a set of lighting directions to properly
sample discrete set of the reflection kernel R(θ, φ) of an object. The sampling theorem tells us that spherical harmonics
transformation of a band-limited reflection kernel with bandwidth B, that is Rlm = 0 (l ≥ B), can be computed byweighted sums of 4B 2 sampled function values.
However, the important issue of how to determine thebandwidth of the reflection kernel of a given object still remains to be solved. There is certainly a situation where it isdifficult to predict the bandwidth of the reflection kernels ofan object, or where only a limited number of samplings ofthe reflection kernel are obtainable due to the limitation of a
hardware set-up used for acquiring input images. In the casewhere samplings of the reflection kernel of an object are insufficient, the obtained coefficients suffer from aliasing. This
leads to undesirable artifacts such as ringing in the reflection
kernel reconstructed from those coefficients.
In the next section, we will carefully consider this issue ofaliasing and extend the method based on the sampling theorem further for reducing the artifacts due to aliasing, by
substituting extended light sources for a point light source
to sample the reflection kernel of a real object.

4 Use of Extended Light Sources
Extended light sources (ELS) may be constructed by illuminating a spherical diffuser with a point light source as
illustrated in Figure 2. The use of ELS brings the following
three advantages:
Functioning as a low-pass filter: ELS have a radiance distribution that is similar to that of the Gaussian function. This
enables extended sources to function as a low-pass filter when
the reflection kernel of an object is sampled under ELS. From
this, we are able to model its appearance without sufferingfrom aliasing.
Adjustable bandwidth: The distribution of ELS can be adjusted by changing the distance from the diffuser to a point
light source. This enables us to adjust the bandwidth of the
ELS in the frequency domain.
Reducing saturation problem: When an object is illuminated by a point light source, image intensities from its specular reflection components often become much greater thanthose from its diffuse reflection components. Under ELS, thegap between image intensities due to specular and diffuse
reflection components are narrowed [11]. This helps avoid
saturation, so that we are able to observe both specular anddiffuse reflection components in the same image. 4
In the following, we consider the reflection kernel of an
object sampled under ELS.
3 This approach is based on the assumption that a point light has unit
radiance and it is sufficiently distant from objects.
4 The used of the multiplexed illumination proposed in [18] is also effective in reducing this dynamic range problem of capturing both diffuse andspecular reflection .

Proceedings of the IEEE International Conference on Computer Vision (ICCV 2005), pp. 325-332, October 2005

E

R'

R

*
θ (degrees)

Extended Source Radiance

Reflection Kernel

Modified Reflection Kernel

Figure 3. Modified Reflection Kernel obtainedby convolving the original reflection kernelR(θ, φ) with the distribution of the ELS E(θe ).
Figure 2. ELS are constructed by a spherical
diffuser and a point light source [11].

4.1 Radiance Distribution of ELS
First, the radiance distribution of ELS needs to be provided. The radiance distribution of ELS is symmetric withrespect to the point source direction and therefore has no azimuth dependence around the point source direction. Accordingly, the radiance of the inner surface of the diffuser E may
be represented as a function of the elevation angle θe , that isE(θe , φe ) = E(θe ). Here θe represents the elevation angle
defined with respect to the direction of the point light source
as shown in Figure 2.
Then E(θe ) are computed from the analytic formula derived in [11] as
E(θe ) =CP [(S + H) cos θe − S]

3[(S + H − S cos θe )2 + (S sin θe )2 ] 2 ,(5)where P denotes the radiance of the point source and C is aconstant representing the proportionality between the irradiance of the outer surface of the diffuser to the radiance of its
inner surfaces. As illustrated in Figure 2, S is the radius of
the spherical diffuser, H is the distance from the the diffuser’s
surface to the point light source.5
Here the surface points on the diffuser within the rangeof θe < α can receive energy from the point light source,
and the effective range α is determined by the relationships
S
between S and H as α = cos−1 ( S+H).

4.2 Reflection Kernel Sampled under ELS
Next, let us consider a reflection kernel R(θ, φ) sampled
as an observed brightness of an object surface when it is illuminated by ELS whose center is located at the direction(θ, φ).
Since radiance of the ELS are distributed over the spherical surface of the diffuser, the modified reflection kernel seen
under ELS, denoted as R (θ, φ) is determined by computing
the integral of the scene radiance resulting from illuminationof the entire surface of the extended source as π  2πR(Mθφ (θe , φe ))E(θe ) sin θe dθe dφe , (6)R (θ, φ) =00

5 This formula is based on the condition that the spherical diffuser is ideal

and thus incident energy is scattered equally in all directions.

where Mθφ (·) is a rotation operator that rotates the distribution of the ELS E(θe ) so that its center (θe = 0) is locatedat the direction (θ, φ). In other words, the modified reflection kernel R (θ, φ) is determined by convolving the originalreflection kernel R(θ, φ) with the radiance distribution of the
ELS E(θe ) as illustrated in Figure 3.
R(Mθφ (θe , φe )) and E(θe ) in (6) can be expanded as a linear combination of spherical harmonics, and (6) is rewritten
in terms of spherical harmonics as

R (θ, φ) =

∞ 
l


l
Rlm El Dm,0(θ)eImφ .

(7)

l=0 m=−lMathematical derivation of (7) is shown in the Appendix.
Rlm and El represent coefficients in the spherical harmonic
expansion of the reflection kernel R(θ, φ) and ELS E(θe )respectively, and the matrix Dl tells us how to compute a rotated spherical harmonic as a linear combination of all thespherical harmonics of the same order l.
The important point to note in (7) is that the modified reflection kernel R (θ, φ) is computed as a product of the coefficient Rlm and El . This indicates that the modified reflection
kernel R (θ, φ) is band-limited with the lower bandwidth between R(θ, φ) and E(θe ). Fortunately, it is not difficult toadjust the bandwidth of the ELS.

4.3 Adjusting Bandwidth of ELS
As has been noted above, ELS used in our approach havea distribution similar to that of the Gaussian function, andtheir distribution and effective range α are determined by theradius S of a spherical diffuser and the distance H from thediffuser surface to a point light source as shown in (5). In
other words, the closer the point light source is positioned tothe diffuser, the narrower its range α becomes. This propertyof Extended Light Sources enables us to specify their bandwidth.
The Fourier transform of the Gaussian function is known
to lead to another Gaussian function in the frequency domain.
In addition, the standard deviation σf of the Gaussian function in the frequency domain is known to become inversely
proportional to the standard deviation σs of the Gaussianfunction in the spatial domain. That is σf = σ1s . Similarly,

Proceedings of the IEEE International Conference on Computer Vision (ICCV 2005), pp. 325-332, October 2005

1

0
1

Spatial domain
α = 30E(ǰe’)

ǰe’ 60 40 20

0

20

40

60

80

α = 40E(ǰe’)

1Frequency domain
El

0

r 0

1

El

2

20

40

60

80

100 120 140

2

an object by ELS with a properly adjusted range α. Here we
employ the efficient sampling theorem in spherical harmonics
proposed in [9].
This sampling theorem tells us that spherical harmonics
transformation of a band-limited function with bandwidth B(Flm = 0 (l ≥ B)), can be computed by weighted sums of2B 2 − B sampled function values based on Gaussian quadrature [13] asFlm =B−1 2B−2
2π  
wj f (θj , φk )Ylm (θj , φk ),2B − 1 j=0

(8)

k=0

0
1

0

ǰe’ 60 40 20

020

E(ǰe’)

ǰe’ 60 40 20

0

20

80

0

r 0

α = 50

1

El

0

r 0

40

40

60

60

80

20

40

60

80

100 120 140

2

20

40

60

80

100 120 140

Figure 4. Distribution of ELS (left: spatial domain, right: frequency domain).
we can safely say that the Gaussian function defined on theunit sphere also results in a half-Gaussian distribution of thecoefficients, that is a Gaussian distribution with a half range,
in its spherical harmonic expansion.
To see how bandwidth of ELS changes depending on their
α values, we synthetically provide several radiance distributions of ELS with different α values from (5). The coefficients El and the corresponding α of those extended sources
are shown in Figure 4. In this figure, spherical harmonic coefficients for given degree l and order m are represented using
a single index r = l2 + l + m. The horizontal axis represents
the index r, and the vertical axis represents the computed coefficients El .
In each graph in this figure, the left side of a red arrowcorresponds to the coefficients up to the degree l = 8. Theenergy captured by spherical harmonics up to the degree l =8, denoted as P8 are also computed by sums of the squaresof their respective coefficients divided by the total squaredenergy of the transformed function. In this figure, we clearly
see that P8 becomes closer to 100% as their α values increase
from 30 to 50 degrees. Especially in the case of α = 50
degrees, more than 99% of the total energy is captured by thespherical harmonics up to the degree l = 8, so it is reasonable
to assume that the extended source generated with α = 50
degrees is band limited with bandwidth l = 8. As shown inthis example, the bandwidth of ELS can be set by adjusting
their effective range α.

5 Modeling Appearance by using ELS
Based on the analysis provided in the previous section, we
propose a novel scheme for sampling the reflection kernel ofwhere the weights wj are weights of Gaussian quadrature, θj
2πk
are Gaussian nodes in cos θ, and φk = (2B−1)
are equally
sampled in azimuth [9, 12].
In order to avoid aliasing caused by insufficient samplingof the appearance of an object, we observe its reflection kernel under ELS whose bandwidth is properly adjusted to B.
By doing so, the modified reflection kernel R (θ, φ) of the object sampled under the ELS also becomes band-limited withthe same bandwidth B as the ELS, and this enables us tomodel the appearance of the object from a set of sampledR (θj , φk ) without suffering from aliasing caused due to insufficient sampling of the original reflection kernel R(θ, φ).

6 Experimental Results: Synthetic and Real6.1 Synthetic Data
The reflection kernel R(θ, φ) of several surface materials
are synthetically provided based on the Ward isotropic reflection model [22] with known reflection parameters as shown in
Table 1.6 Here the surface normal and viewing direction for
the reflection kernel are set at the direction (θn = 0, φn = 0)
and (θo = 45, φo = 0) respectively.
The ELS used in this experiment are generated with α =50 degrees whose bandwidth is equivalent to B = 9. This
makes the modified reflection kernel R (θ, φ) band-limitedwith B = 9. Accordingly, the spherical harmonic coeffim
cients R l of the band-limited function R (θ, φ) were obtained from a properly sampled discrete set of the function
values (153 samplings for B = 9 in (8)).
The right columns in Figure 5, and 6 show the obtainedreflection kernel for glossy gray paper and lightly brushed
aluminum surfaces respectively. b) shows computed coeffim
cients R l , and d) and f) show the distribution of R (θ, φ)mreconstructed from the obtained R l in (1) up to the degree7
l = 8.
For reference, the left columns of these figures show thecoefficients and distribution of the original reflection kernelR(θ, φ). a) shows its coefficients computed from 100000
6 K and K are constants for the diffuse and specular reflection compos
dnents, and σ is the standard deviation of the surface slope.
7 d) shows R (θ, φ) scanned in the line of {θ|0 ≤ θ ≤ π/2}, {φ|0or π}, and f) shows the upper half of the reflection kernel visualized in apolar coordinate system with radius indicating {θ|0 ≤ θ ≤ π/2}, and angle
indicating {φ|0 ≤ φ < 2π} .

Proceedings of the IEEE International Conference on Computer Vision (ICCV 2005), pp. 325-332, October 2005

1.2

Table 1. Reflection parameters.
Material
glossy gray paper
lightly brushed aluminum

kd
.29
.15

ks
.083
.19

1

σ
.082
.088

samplings of R(θ, φ), and c) and e) show its distribution computed by substituting the reflection parameters into the Ward
reflection model.
Comparing a) with b), while the magnitude of the coefficients is different because of the multiplication with the coefficients of the ELS whose distribution is similar to that of amhalf Gaussian function, the recovered R l go up and down inmthe same manner as Rl if we see their distributions locally.
In Figure 7 a) and b), we also provide the low-frequency appearance of the original reflection kernel for reference by substituting the coefficients Rlm up to degree l = 8 into (1). In
other words, all coefficients with the index l > 8 of Rlm aretruncated in this case. Comparing f) with the low-frequency
appearance of the original reflection kernel in Figure 7 a), themodified reflection kernel reconstructed by our method provides a good representation of the low-frequency appearanceof the original reflection kernel, although high-frequency appearance such as the specular peak in e) is blurred due to theapproximation of the original reflection kernel up to the degree l = 8.
It is worth noting that almost no undesirable artifacts appear in the reconstructed reflection kernel R (θ, φ) in f). Thisshows that the proposed method succeeded in computing themcoefficients R l of the modified reflection kernel from thediscrete 153 samplings of R (θ, φ) without suffering fromaliasing due to insufficient sampling of its appearance. In
contrast, as Westin et al. pointed out in [23], simply truncating all coefficients Rlm with l > 8 resulted in undesirable
artifacts such as ringing in the reconstructed reflection. Thisringing is called the Gibbs phenomenon and is shown in Figures 7 a) and b).
In order to avoid the Gibbs phenomenon, Westin et al. progressively reduced the magnitude of the coefficients according to a half Gaussian distribution of an empirically determined width. Fortunately, the coefficients of the ELS used in
our method have a distribution that is similar to that of a half
Gaussian distribution, as shown in Figure 4. It follows fromthis that the use of ELS is desirable not only as a method ofmodifying the original reflection kernel to be band-limited,
but also for reducing the number of artifacts caused by thetruncation of the coefficients with index l ≥ B of the originalreflection kernel. 8

Spherical harmoniccoefficients
1.2

m

Rl

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

0

-0. 2

-0. 2

-0.4

-0.4

-0. 6

0

1.0
0.9

R' lm

r

10 20 30 40 50 60 70 80

-0. 6

0

r

10 20 30 40 50 60 70 80

a)b)Distribution of reflection
kernel
1.0

R

0.9

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

R'

0

150

100

50

0

50

100

150

150

100

50

0

θ (degrees)c)d)Visualization of reflection kernel

e)

50

100

150

θ (degrees)f)Figure 5. Glossy gray paper : original reflection kernel (left), recovered modified reflection
kernel (right).
1.2
1Spherical harmoniccoefficients
1.2

m

Rl

R' lm

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

0

-0. 2

-0. 2

-0.4

-0.4

-0. 6

0

1.0
0.9

10 20 30 40 50 60 70 80

r

-0. 6

0

10 20 30 40 50 60 70 80

a)b)Distribution of reflection
kernel
1.0

R

0.9

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

r

R'

0

150

100

50

0

50

100

150

150

100

50

0

θ (degrees)c)d)Visualization of reflection kernel

50

100

150

θ (degrees)

6.2 Real Data
Real images of abalone shellfish were taken under thephysically constructed ELS apparatus shown in Figure 1.
8 R (θ, φ) is computed as a product of Rm , E , and D l in (7). Sincell
El and D l are computable numbers, we are able to recover Rm
l of the original reflection kernel from the set of modified reflection kernel R (θ, φ) if

necessary.

e)f)Figure 6. Lightly brushed aluminum : originalreflection kernel (left), recovered modified reflection kernel (right).

Proceedings of the IEEE International Conference on Computer Vision (ICCV 2005), pp. 325-332, October 2005

glossy gray paper
1.0
0.9

R

0.8
0.7
0.6
0.5

l,m) = (0, 0)(0, -1)(1, 1)(1, 1)(2, -1)(2, 0)(2, 1)(2, 2)(2, -2)

0.4
0.3
0.2
0.1
0

150

100

lightly brushed aluminum
1.0
0.9

50

0

50

100

150

θ (degrees)(a) Obtained harmonic images: positive values are shownin green; negative values are shown in red.

R

0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

150

a)

100

50

0

b)

50

100

150

θ (degrees)

Figure 7. Reconstructed reflection kernel fromthe coefficients Rlm of the original reflection
kernel up to the degree l = 8.
Abalone shellfish is famous for its interesting structural colors which alter greatly depending on viewing direction andillumination conditions. In this set-up, 9 halogen lamps werepositioned at Gaussian nodes in elevation, and an array ofthese light sources were mounted on a turntable and rotatedaround the spherical diffuser (an acrylic globe with a diame2πk
ter of 35cm) by φk = (18−1), (k = 0, . . . , 16) degrees in azimuth. Here the number of point light sources in elevation
indicates that a modified reflection kernel has to be bandlimited with B = 9. Accordingly the distance between the
diffuser and the point light sources was adjusted so that the
bandwidth of the constructed ELS was set to B = 9.
In total, 153 input images of the abalone shellfish weretaken to sample its reflection kernel at each grid point (θj ,mφk ). Then coefficients R l of this reflection kernel werecomputed up to the degree l = 8 by substituting the observedreflection kernel R (θj , φk ) into (8). The first nine harmonic
images obtained by our method are shown in Figure 8 (a).
Figure 8 (b) shows the appearance of the abalone shellfish
synthesized from (4) under natural illumination conditions
provided as high-dynamic range light probe measurements by[2]. In this figure, the synthesized appearance of the abalone
shellfish significantly changes depending on the characteristics of the given illumination distributions, and this showsthat the complex appearance of its structural colors are well
represented by a set of basis images obtained by our method.
In addition, the synthesized appearance of the shellfish iscompared with its real appearance as seen under a normal
lighting condition in our laboratory in Figure 1.9 In this figure, the synthesized appearance highly resembles the real image. As has been noted before, in the case of a highly specular surface, image intensities from specular reflection components tend to be much greater than those from diffuse reflection components. This is the reason why image intensities in
9 Coefficients Lm of this lighting condition are computed from an omnil
directional image of the scene taken by a camera with a fish-eye lens.

(b) Synthesized images under natural Illumination conditions

Figure 8. (a) Obtained harmonic images. (b)Synthesized images under natural illumination
conditions.
the real image become saturated in some specular regions.
In general, the limited dynamic range of an image takenwith one shutter speed makes it difficult to model the appearance of highly specular surfaces. In contrast, the use of ELS
contributes to reduce the high contrast between image intensities from both specular and diffuse components. This helps
us to observe and model the appearance of highly specularsurfaces from images taken at a single shutter speed.

7 Conclusions
In this paper we have demonstrated the effectiveness of using Extended Light Sources (ELS) for modeling the appearance of an object under varying illumination. The use of ELS
for modeling the appearance of objects has the following advantages: (1) ELS have the ability to function as a low-passfilter for sampling objects’ appearance, so basis images of an
object can be obtained without suffering from aliasing caused
by insufficient sampling of its reflection kernel, (2) ELS can
reduce the high contrast between image intensities from both
specular and diffuse components. This helps us observe andmodel the appearance from images taken at a single shutter
speed, and (3) ELS can minimize undesirable artifacts resulting from the truncation of spherical harmonic coefficients ofhigher degrees. These advantages help provide basis imagesof an object from a limited number of samplings of its appearance. Our method can adequately synthesize the appearance of an object up to a certain frequency. However, objectsof high bandwidth such as a mirror illuminated by a point

Proceedings of the IEEE International Conference on Computer Vision (ICCV 2005), pp. 325-332, October 2005

light source are difficult to be adequately modeled. A future
research direction of this work would be to integrate othermodeling techniques with the objective of recovery of an object original reflection kernel. Also, it would be interesting toinvestigate the potential of other types of light sources serving as a low-pass filter when the appearance of an object issampled under them.

Acknowledgments
This research was supported in part by Grands-in-Aid for Scientific Research (No.14-08417) from the Ministry of Education, Culture, Sports, Science and Technology of Japan, and the PRESTO
program of Japan Science and Technology Corporation.

[16] R. Ramamoorthi and P. Hanrahan, “An efficient representation for irradiance environment maps,” Proc. SIGGRAPH 01, pp. 497-500, 2001.
[17] I. Sato, T. Okabe, Y. Sato, and K. Ikeuchi, “Appearance sampling for
obtaining a set of basis images for variable illumination,” Proc. IEEE
Int’l Conf. Computer Vision, pp. 800-807, 2003.
[18] Y. Schechner, S. Nayar, and P. Belhumeur, “A Theory of Multiplexed
Illumination,” Proc. IEEE Int’l Conf. Computer Vision, pp. 808-815,
2003.
[19] K. Shim and T. Chen, “Efficient representation of lighting patterns forimage-based relighting,” Proc. Picture coding symposium, 2004.
[20] A. Shashua, “On photometric issues in 3D visual recognition from asingle image,” Int. J. Computer Vision, vol. 21, pp. 99-122, 1997.
[21] P. Sloan, J. Kautz, and J. Snyder, “Precomputed radiance transfer for
real-time rendering in dynamic, low-frequency lighting environments,”
SIGGRAPH 02, pp. 527-536, 2002.

References[22] G. J. Ward, “Measuring and modeling anisotropic reflection,” Proc.
SIGGRAPH ’92, pp. 265-272. 1992.

[1] R. Basri and D. Jacobs, “Lambertian Reflectance and Linear Subspaces,” Proc. IEEE Intl. Conf. Computer Vision 01, pp.383-389, 2001.

[23] S. Westin, J. Arvo, K. Torrance, “Predicting Reflectance Functions
from Complex Surfaces, ” Proc. SIGGRAPH ’92, pp. 255-264, 1992.

[2] P. Debevec, “Rendering synthetic objects into real scenes: Bridging
traditional and image-based graphics with global illumination withhigh dynamic range photography,” Proc. SIGGRAPH 98, pp. 189-198,
1998.

[24] A. Yuille, D. Snow, R. Epstein, and P. Belhumeur, “Determining generative models of objects under varying illumination: Shape and albedo
from multiple images using SVD and integrability,” Int. J. Computer
Vision, vol. 35, no. 3, pp. 203-222, 1999.

[3] R. Epstein, P. Hallinan, and A. Yuille, “5+/-2 eigenimages suffice:An empirical investigation of low-dimensional lighting models,” Proc.
IEEE Workshop on Physics-Based Modeling in Computer Vision, pp.
108-116, 1995.

[25] L. Zhao and Y. Yang, “Theoretical analysis of illumination in PCAbased vision systems,” Pattern Recognition, vol. 32, pp. 547-564, 1999.

[4] A. Georghiades, D. Kriegman, and P. Belhumeur, “Illumination cones
for recognition under variable lighting: Faces,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 52-59, 1998.
[5] A. Georghiades, D. Kriegman, and P. Belhumeur, “From few to many:
Generative models for recognition under variable pose and illumination,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23,
no. 6, pp. 643-660, 2001.

Appendix
A modified reflection kernel R (θ, φ) is described by
Z π Z 2πR(Mθφ (θe , φe ))E(θe ) sin θe dθe dφe ,R (θ, φ) =0[8] T. Malzbender, D. Gelb, and H. Wolters, “Polynomial texture maps,”
Proc. SIGGRAPH 01, pp. 519-528, 2001.
[9] M. J. Mohlenkamp, “A Fast Transform for Spherical Harmonics,” J.
Fourier Analysis and Applications, vol. 5(2/3), pp.159-184, 1999.
[10] H. Murase and S. Nayar, “Visual learning and recognition of 3-D objects from appearance,” Int. J. Computer Vision, vol. 14, no. 1, pp.
5-24, 1995.
[11] S. K. Nayar, K. Ikeuchi, and T. Kanade, “Determining shape and reflectance of hybrid surfaces by photometric sampling,” IEEE Trans.
Robotics and Automation, vol. 6, no. 4, pp. 418-43, 1990.
[12] T. Okabe, I. Sato, and Y. Sato, “Spherical Harmonics vs. Haar
Wavelets: Basis for Recovering Illumination from Cast Shadows,”
Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp.I-5057, 2004.
[13] W. H. Press, B. P. Flannery, S. A. Teukolsky, W. T. Vetterling, Numerical Recipes in C: The Art of Scientific Computing, Cambridge University Press, Cambridge, 1988.
[14] R. Ramamoorthi and P. Hanrahan, “A signal-procession framework forinverse rendering”, Proc. SIGGRAPH 01, pp.117-128, 2001[15] R. Ramamoorthi and P. Hanrahan, “On the relationship between radiance and irradiance: Determining the illumination from images of aconvex lambertian object,” J. Optical Society of America A, vol. 18,
no. 10, pp. 2448-2459, 2001.

(9)

R(Mθφ (θe , φe )) and E(θe ) can be expanded as a linear combination of
spherical harmonics:
E(θe )

[6] P. Hallinan, “A low-dimensional representation of human faces for arbitrary lighting conditions,” Proc. IEEE Conf. Computer Vision and
Pattern Recognition, pp. 995-999, 1994.
[7] K. C. Lee, J. Ho, and D. Kriegman, “Nine points of light: Acquiring
subspaces for face recognition under variable lighting,” Proc. IEEE
Conf. Computer Vision and Pattern Recognition 01, pp.519-526, 2001.

0

∞
X

=

En Yn0 (θe )

(10)

n=0

R(Mθφ (θe , φe ))

∞ X
l
X

=

φ 
m

Rm
l Yl (Mθ (θe , φe )),(11)

l=0 m=−lwhere Rml and En denote coefficients in their spherical harmonic expansion. Then a rotation formula for spherical harmonics is given in [14] asP
lImφ Y m (θ  , φ ), where theYlm (Mθφ (θe , φe )) = lm =−l Dm,m (θ)eeelterm eImφ considers the rotation about φ, and the matrix D l tells us how
to compute a rotated spherical harmonic as a linear combination of all thespherical harmonics of the same order l.
Substituting (10) and (11), (9) becomesR (θ, φ) =

∞ X
∞ X
l
X

l
X

n=0 l=0 m=−l m =−l

Z π Z 2π
0

0

l
Imφ
Rm
l En Dm,m (θ)e

Ylm (θe , φe )Yn0 (θe ) sin θe dθe dφe .

Then the orthonormality of the spherical
R π R 2π m   0 
Yl (θe , φe )Yn (θe ) sin θe dθe dφe =0 0foreR (θ, φ) =

(12)

∞ X
∞ X
l
X

l
X

n=0 l=0 m=−l m =−lharmonics tells usδln δm 0 , and there-

l
Imφ
Rm
δln δm 0 .
l En Dm,m (θ)e(13)
Finally, from the characteristics of Kronecker delta (δij = 1 if i = j, andδij = 0 if i = j), the modified reflection kernel is obtainedR (θ, φ) =

∞ X
l
X

l
Imφ
Rm
.
l El Dm,0 (θ)e

l=0 m=−l

Proceedings of the IEEE International Conference on Computer Vision (ICCV 2005), pp. 325-332, October 2005

(14)