IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. Y, NO. Y, MON 2002

0

Illumination from Shadows
Imari Sato, Yoichi Sato, and Katsushi Ikeuchi
Abstract— In this paper, we introduce a method for recovering an illumination distribution of a scene from image brightness inside shadows castby an object of known shape in the scene. In a natural illumination condition, a scene includes both direct and indirect illumination distributed
in a complex way, and it is often difficult to recover an illumination distribution from image brightness observed on an object surface. The main
reason for this difficulty is that there is usually not adequate variation in
the image brightness observed on the object surface to reflect the subtle
characteristics of the entire illumination. In this study, we demonstrate theeffectiveness of using occluding information of incoming light in estimating
an illumination distribution of a scene. Shadows in a real scene are causedby the occlusion of incoming light, and thus analyzing the relationships between the image brightness and the occlusions of incoming light enables us
to reliably estimate an illumination distribution of a scene even in a complex illumination environment. This study further concerns the following
two issues that need to be addressed. First, the method combines the illumination analysis with an estimation of the reflectance properties of a shadow
surface. This makes the method applicable to the case where reflectanceproperties of a surface are not known a priori and enlarges the variety ofimages applicable to the method. Second, we introduce an adaptive sampling framework for efficient estimation of illumination distribution. Using
this framework, we are able to avoid unnecessarily dense sampling of theillumination and can estimate the entire illumination distribution more efficiently with a smaller number of sampling directions of the illumination
distribution. To demonstrate the effectiveness of the proposed method, we
have successfully tested the proposed method by using sets of real images
taken in natural illumination conditions with different surface materials of
shadow regions.
Keywords—computer vision, physics based vision, illumination distribution estimation

I. I NTRODUCTION
The image brightness of a three-dimensional object is thefunction of the following three components: the distribution oflight sources, the shape of a real object; and the reflectance ofthat real object surface [8], [9]. The relationship among themhas provided three major research areas in physics-based vision: shape-from-brightness (with a known reflectance and illumination), reflectance-from-brightness (with a known shape andillumination), and illumination-from-brightness (with a known
shape and reflectance).
The first two kinds of analyses, shape-from-brightness andreflectance-from-brightness, have been intensively studied using
the shape from shading method [10], [12], [11], [24], as well as
through reflectance analysis research [13], [22], [7], [1], [15],[18], [30], [35].
In contrast, relatively limited amounts of research have been
conducted in the third area, illumination-from-brightness. Thisis because real scenes usually include both direct and indirect illumination distributed in a complex way and it is difficult to analyze characteristics of the illumination distribution of the scenefrom image brightness. Most of the previously proposed approaches were conducted under very specific illumination conI. Sato is with Graduate School of Interdisciplinary Information Studies, The
University of Tokyo, 7-3-1 Hongo, Bunkyo, Tokyo 113-0033, Japan. E-mail:imarik@iis.u-tokyo.ac.jp .
Y. Sato and K. Ikeuchi are with Institute of Industrial Science, The Universityof Tokyo, 4-6-1 Komaba, Meguro, Tokyo 153-8505, Japan. E-mail: {ysato,ki}@iis.u-tokyo.ac.jp .

ditions, e.g., there were several point light sources in the scene,
and those approaches were difficult to be extended for more natural illumination conditions [11], [13], [30], [33], [36], or multiple input images taken from different viewing angles were necessary [17], [23].
In this study, we present a method for recovering an illumination distribution of a scene from image brightness observedon an object surface in that scene. To compensate for the insufficient information for the estimation, we propose to use occluding information of incoming light caused by an object inthat scene as well as the observed image brightness on an object surface. More specifically, the proposed method recovers
the illumination distribution of the scene from image brightness
inside shadows cast by an object of known shape in the scene.
Shadows in a real scene are caused by the occlusion of incoming light, and thus contain various pieces of information
about the illumination of the scene. However, shadows havebeen used for determining the 3D shapes and orientations of an
object which casts shadows onto the scene [19], [16], [32], [3],while very few studies have focused on the the illuminant information which shadows could provide. In our method, image
brightness inside shadows is effectively used for providing distinct clues to estimate an illumination distribution.
The proposed approach estimates an illumination distributionof a scene using the following procedures. The illumination distribution of a scene is first approximated by discrete sampling ofan extended light source; whole distribution is represented as aset of point sources equally distributed in the scene. Then thisapproximation leads each image pixel inside shadows to provide a linear equation with radiance of those light sources as
unknowns. Finally, the unknown radiance of each light source
is solved from the obtained set of equations. In this paper, we refer to the image with shadows as the shadow image, to the object
which casts shadows onto the scene as the occluding object, andto the surface onto which the occluding object casts shadows as
the shadow surface.
The assumptions that we made for the proposed approach are
as follows.
• Known geometry: the 3D shapes of both the occluding object and the occluding surface are known as well as theirrelative poses and locations.
• Distant illumination: we assume that light sources in thescene are sufficiently distant from the objects, and thus alllight sources project parallel rays onto the object surface.
Namely, the distances from the objects to the light sources
are not considered in the proposed approach.
• No interreflection: the proposed method does not take into
account interreflection between the shadow surface and theoccluding object on the assumption that there is no severeinterreflection between them and no multiple scattering of
the interreflected rays from them to the scene. As a consequence, objects with darker color and weaker specularity

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. Y, NO. Y, MON 2002

are preferred as occluding objects since they do not significantly act as secondary light sources, and the method cannot handle a scene consisting of shiny occluding objects.
This study further concerns the following two issues thatneed to be considered when we tackle this illumination-frombrightness problem. The first issue is how to provide reflectanceproperties of the shadow surface in cases where they are notgiven a priori. Since it is a common situation that reflectanceproperties of a surface are not known, a solution to this problem is required to enlarge the variety of images to which theproposed method can be applied. In our approach, instead ofassuming any particular reflectance properties of the surface inside shadows, both the illumination distribution of the scene andthe reflectance properties of the surface are estimated simultaneously, based on an iterative optimization framework.
The second issue is how to efficiently estimate an illumination distribution of a scene without losing the accuracy of theestimated illumination distribution. To capture the subtle characteristics of the illumination distribution of a scene, the entire
distribution must be discretized densely, and this makes the solution exceedingly expensive in terms of processing time andstorage requirements. To satisfy both the accuracy and the efficiency claims, we introduce an adaptive sampling framework
to sample the illumination distribution of a scene. Using thisadaptive sampling framework, we are able to avoid unnecessarily dense sampling of the illumination and estimate overall illumination more efficiently by using fewer sampling directions.
The rest of the paper is organized as follows. We describe
how we first obtain a formula which relates an illumination distribution of a scene with the image irradiance of the shadow image in Section II and Section III. The formula will later be used
as a basis for estimating both the illumination distribution of areal scene and the reflectance properties of the shadow surface.
Using this formula, we explain how to estimate an illumination
radiance distribution from the observed image irradiance of ashadow image in Section IV. In this section, we consider thefollowing two cases separately: (1) where the reflectance properties of the shadow surface are known, (2) where those properties are not known. We also introduce an adaptive samplingframework for efficient approximation of the entire illumination,and show experimental results of the proposed method applied
to images taken in natural illumination conditions in Section V.
Finally, we discuss some of the related work proposed in thefield of computer graphics in Section VI, and present concluding remarks and future research directions in Section VII. This
paper is an expanded and more detailed version of the works we
presented in [27], [28].
II. R ELATING I LLUMINATION R ADIANCE WITH I MAGEI RRADIANCE
In this section, we describe how we obtain a formula which
relates an illumination distribution of a real scene with the image
irradiance of a shadow image. The formula will later be used asa basis for estimating the illumination distribution of a scene andthe reflectance properties of a shadow surface.
First, we find a relationship between the illumination distribution of a real scene and the irradiance at a surface point in

1

Fig. 1. (a) the direction of incident and emitted light rays (b) infinitesimal patchof an extended light source, (c) occlusion of incoming lightthe scene. 1 To take illumination from all directions into account, let us consider an infinitesimal patch of the extended lightsource, of a size δθi in polar angle and δφi in azimuth as shownin Fig. 1.
Seen from the center point A, this patch subtends a solid angleδω = sinθi δθi δφi . Let L0 (θi , φi ) be the illumination radiance
per unit solid angle coming from the direction (θi , φi ); then theradiance from the patch is L0 (θi , φi )sinθi δθi δφi , and the total
irradiance of the surface point A is [9]E= π  π2−π0

L0 (θi , φi )cosθi sinθi dθi dφi(1)Then occlusion of the incoming light by the occluding object isconsidered as π  π2L0 (θi , φi )S(θi , φi )cosθi sinθi dθi dφi
(2)
E=
−π

0

where S(θi , φi ) are occlusion coefficients; S(θi , φi ) = 0
if L0 (θi , φi ) is occluded by the occluding object; Otherwise
S(θi , φi ) = 1 (Fig. 1 (c)).
Some of the incoming light at point A is reflected toward theimage plane. As a result, point A becomes a secondary lightsource with scene radiance.
The bidirectional reflectance distribution function (BRDF)
f (θi , φi ; θe , φe ) is defined as a ratio of the radiance of a surface as viewed from the direction (θe , φe ) to the irradiance resulting from illumination from the direction (θi , φi ). Thus, byintegrating the product of the BRDF and the illumination radiance over the entire hemisphere, the scene radiance Rs(θe , φe )viewed from the direction (θe , φe ) is computed as
Rs(θe , φe ) = π  π2−π0f (θi , φi ; θe , φe )L0 (θi , φi )S(θi , φi )cosθi sinθi dθi dφi(3)

Finally, the illumination radiance of the scene is related withimage irradiance on the image plane. Since what we actuallyobserve is not image irradiance on the image plane, but rather arecorded pixel value in a shadow image, it is necessary to consider the conversion of the image irradiance into a pixel valueof a corresponding point in the image. This conversion includes
1 For a good reference of the radiometric properties of light in a space, see[21].

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. Y, NO. Y, MON 2002

several factors such as D/A and A/D conversions in a CCD camera and a frame grabber.
Other studies concluded that image irradiance was proportional to scene radiance [9]. In our work, we calibrate a linearityof the CCD camera by using a Macbeth color chart with knownreflectivity so that the recorded pixel values also become proportional to the scene radiance of the surface. From (3) the pixel
value of the shadow image P (θe , φe ) is thus computed as
P (θe , φe ) = k π  π2−π0A. Known Reflectance Properties(4)

where k is a scaling factor between scene radiance and a pixel
value. Due to the scaling factor k, we are able to estimate unknown L0 (θi , φi )(i = 1, 2, · · · ·, n) up to scale. To obtain the scale
factor k, we need to perform photometric calibration between
pixel intensity and the physical unit (watt/m2 ) for the irradiance.
III. A PPROXIMATION OF I LLUMINATION D ISTRIBUTION
WITH D ISCRETE S AMPLING
In our implementation of the proposed method, to solve forthe unknown radiance L0 (θi , φi ) which is continuously distributed on the surface of the extended light source from therecorded pixel values of the shadow surface, the illumination
distribution is approximated by discrete sampling of radianceover the entire surface of the extended light source. This can beconsidered as representing the illumination distribution of thescene by using a collection of light sources with an equal solid
angle. As a result, the double integral in (4) is approximated asnFig. 2. Each pixel provides a linear equation.

f (θi , φi ; θe , φe )L0 (θi , φi )S(θi , φi )cosθi sinθi dθi dφi

P (θe , φe ) =2

f (θi , φi ; θe , φe )L(θi , φi )S(θi , φi )cosθi ωi

i=1

(5)
where n is the number of sampling directions, L(θi , φi ) is the illumination radiance per unit solid angle coming from the direction (θi , φi ), which also includes the scaling factor k between
scene radiance and a pixel value, and ωi is a solid angle for the
sampling direction (θi , φi ).
For instance, node directions of a geodesic dome can be used
for uniform sampling of the illumination distribution. By using n nodes of a geodesic dome in a northern hemisphere asa sampling direction, the illumination distribution of the scene
is approximated as a collection of directional light sources distributed with an equal solid angle ω = 2π/n.
IV. E STIMATION OF R ADIANCE D ISTRIBUTION BASED ON
R EFLECTANCE P ROPERTIES OF S HADOW S URFACE
After obtaining the formula that relates the illumination radiance of the scene with the pixel values of the shadow image,illumination radiance is estimated based on the recorded pixel
values of the shadow image.
In (5), the recorded pixel value P (θe , φe ) is computed as afunction of the illumination radiance L(θi , φi ) and the BRDF
f (θi , φi ; θe , φi ). Accordingly, in the following sections, we takedifferent approaches, depending on whether BRDF of the surface is given.

A.1 Lambertian
Let us start with the simplest case where the shadow surface
is a Lambertian surface whose reflectance properties are given.
BRDF f (θi , φi ; θe , φe ) for a Lambertian surface is known to
be a constant. From (5), an equation for a Lambertian surface isobtained as
P (θe , φe ) =n

Kd L(θi , φi )S(θi , φi )cosθi ωi

(6)

i=1

where Kd is a diffuse reflection parameter of the surface.
From (6), the recorded pixel value P for an image pixel is givenasn
ai L i(7)
P =i=1

where Li (i = 1, 2, · · · ·, n) are n unknown illumination radiance specified by n node directions of a geodesic dome. As
shown in Fig. 2, the coefficients ai (i = 1, 2, · · · ·, n) represent
Kd S(θi , φi ) cos θi ωi in (6); we can compute these coefficientsfrom the 3D geometry of a surface point, the occluding object
and the illuminant direction. In our examples, we use a modeling tool called the 3D Builder from 3D Construction Company[37] to recover the shape of an occluding object and also thecamera parameters from a shadow image. At the same time, theplane of z = 0 is defined on the shadow surface. 2
If we select a number of pixels, say m pixels, a set of linear
equations is obtained as
Pj =

n


aji Li(j = 1, 2, · · · ·, m)

(8)

i=1

Therefore, by selecting a sufficiently large number of image
pixels, we are able to solve for a solution set of unknown Li ’s. 3
For our current implementation, we solve the problem by using
the linear least square algorithm with non-negativity constraints(using a standard MATLAB function) to obtain an optimal solution with no negative components.
2 It is worth noting that, as long as the 3D shape of the shadow surface is provided in some manner, e.g., by using a range sensor, the shadow surface neednot be a planar surface. It is rather reasonable to think that a curved shadow surface has the potential for providing more variation in aji in (8) than an ordinary
planar surface would since cosθi term of ai in (7) also slightly changes fromimage pixel to pixel. Unless severe interreflection occurs within the surface, ourmethod can be applied to a curved shadow surface.
3 More details of the pixel selection are found in the Appendix.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. Y, NO. Y, MON 2002

It should be noted that each pixel P (θe , φe ) consists of 3 color
bands (R, G, and B) and likewise the diffuse parameter Kd consists of 3 parameters (Kd,R , Kd,G , Kd,B ). Each color band ofL(θi , φi ) is thus estimated separately from the corresponding
band of the image pixel and that of the diffuse parameter Kd .
For the sake of simplicity in our discussion, we explain the proposed method by using L(θi , φi ), P (θe , φe ), Kd and do not referto each of their color bands in the following sections.
We must also draw attention to the importance of shadow
coefficients for recovering illumination distribution of a scenefrom the observed brightness on an object surface. For instance,
consider the case described above where the given shadow surface is a Lambertian surface. If we do not take into accountthe occlusion of including light, the variation of aji in (8) iscaused only by the cosine factor of incoming light direction andthe surface normal direction of the corresponding point on the
shadow surface (cosθji ). As a result, the estimation of illumination distribution by solving the equation (8) for a solution setof unknown Li ’s tends to become numerically unstable.
In contrast, taking into account the occlusion of incoming
light tells us that the variation of aji becomes much larger since
the shadow coefficient S(θi , φi ) in (6) changes significantly, i.e.,either 0 or 1, from point to point on the shadow surface depending on the spatial relationships among the location of thepoint, the occluding object, and the sampling direction of the illumination distribution, as is well illustrated in Lambert’s work
described in [5]. This characteristics of shadows enables us toreliably recover the illumination distribution of the scene frombrightness changes inside shadows.

3

A.3 Experimental Results for Known Lambertian Surface

We have tested the proposed approach using an image with an
occluding object, i.e., shadow image, taken under usual illumination environmental conditions, including direct light sources
such as fluorescent lamps, as well as indirect illumination suchas reflections from a ceiling and a wall (Fig. 3 (b)).
First, an illumination distribution of the scene was estimatedusing the image irradiance inside shadows in the shadow image. Then a synthetic object with the same shape as that of theoccluding object was superimposed onto an image of the scenetaken without the occluding object, which is referred as a surface image, using the rendering method described in [29]. Notethat our algorithm does not require this surface image in the process of illumination estimation in the case where the reflectanceproperties of the shadow surface are given. This surface image is used here as a background image for superimposing thesynthetic occluding object onto the scene.
Synthesized results are shown in Fig. 4 (a), (b), and (c). Also,
we superimposed another synthetic object of a different shape
onto the scene in Fig. 4 (d). The number of nodes of a geodesic
dome used for the estimation is shown under the resulting image.
As we see in Fig. 4, the larger the number of nodes we used,
the more the shadows of the synthetic object resembled those of
the occluding object in the shadow image. Especially in the caseof 520 nodes, the shadows of the synthetic object are nearly indistinguishable from those of the occluding object in the shadow
image; this shows that the estimated illumination distribution
gives a good presentation of that of the real scene. 4
Fig. 5 numerically shows the improvement of the accuracy
A.2 Non-Lambertian surface
obtainedby increasing the number of samplings. Here, the esThe proposed approach may be extended to other reflection
timatedillumination
distribution was evaluated in a different
models as well. The only condition for a model to satisfy is that
scene
condition
where
the occluding object was rotated 45 deit enables us to analytically solve for a solution set of unknown
grees
on
its
z
axis.
The
vertical axis represents the average error
illumination radiance from image brightness.
inpixel
values(0
to255)
inside the shadow regions in the synTake a simplified Torrance-Sparrow reflection model [22],thesized
imagescomparedwith those in the shadow image. The[34] for example; the pixel value of a shadow image P (θe , φe )horizontal
axis
represents
the number of nodes of a geodesic
is computed as
dome
used
for
the
estimation.
The small pictures right next ton
Kd L(θi , φi )S(θi , φi )cosθi ωi +P (θe , φe ) =the plot show error distributions inside shadow regions in thei=1
synthesized images. Darker color represents larger error in a−γ(θi ,φi )2n
1
Ks L(θi , φi )S(θi , φi )ωi
e 2σ2
i=1
pixel value. The average error at the zero sampling of illuminacosθetion radiance represents, for reference, the average pixel value
2
−γ(θ
,φ
)
n
i i
1
2σ2
=
(K
cosθ
+
K
e
)
inside the shadow regions of the shadow image.
d
i
s
i=1
cosθe
In this plot, we see that flattening of the error curve before
S(θi , φi )ωi L(θi , φi )(9)convergence to the exact solutions occurs. This is likely caused
where γ(θi , φi ) is the angle between the surface normal and by the following factors. The first of these is the errors in thethe bisector of the light source direction and the viewing direc- measurements of the 3D shapes of the occluding object and thetion, Kd and Ks are constants for the diffuse and specular re- shadow surface. The second factor is the errors in the given
flection components, and σ is the standard deviation of a facet reflectance properties of the shadow surface; there is some possibility that the shadow surface is not a perfect Lambertian surslope of the Torrance-Sparrow reflection model.
From (9), we obtain a linear equation for each image pixel face. The third factor is related to our lighting model that apwhere L(θi , φi )(i = 1, 2, · · · ·, n) are unknown illumination ra- proximates the illumination distribution of a scene by discrete−γ(θi ,φi )2
sampling of an extended light source on the assumption that12σ2e
)S(θi , φi )ωi (i =diance, and (Kd cosθi + Ks cosθe
4 We see some “stair-casting” artifacts inside the synthesized shadows that re1, 2, · · · ·, n) are known coefficients.
sultfrom our approximation of continuous illumination distribution by discrete
Again, if we use a sufficiently large number of pixels for the
sampling of its radiance. One approach to reduce this artifact is to increase theestimation, we are able to solve for a solution set of unknown number of point light sources used for rendering, whose radiance is given byinterpolating the estimated radiance Li of the neighboring light sources.
illumination radiance L(θi , φi )(i = 1, 2, · · · ·, n) in this case.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. Y, NO. Y, MON 2002

4

Fig. 3. Input images : (a) surface image (b) shadow image

Fig. 5. Error Analysis: The vertical axis represents the average error in pixel
values (0 to 255) inside the shadow regions in the synthesized images comparedwith those in the shadow image. The horizontal axis represents the number ofnodes of a geodesic dome used for the estimation. The small pictures right next
to the plot show error distributions inside shadow regions in the synthesizedimages. Darker color represents larger error in a pixel value.

B.2 Non-Uniform Surface Material

Fig. 4. Synthesized images: known reflectance property

light sources in the scene are sufficiently distant from the objects. The input image used in this experiment was taken in anindoor environment, this assumption might not perfectly hold.
The last factor concerns our assumption that there is no interreflection between the occluding object and the shadow surface.

B. Unknown Reflectance Properties
B.1 Uniform Surface Material
There certainly may be a situation where reflectance properties of a surface are not known a priori, and we need to somehow
provide those properties in advance. To cope with this situation,
we combine the illumination analysis with an estimation of thereflectance properties of the shadow surface. More specifically,
we estimate both the illumination distribution of the scene andthe reflectance properties of the surface simultaneously, basedon an iterative optimization framework. We will later explain
details of this iterative optimization framework in conjunction
with the adaptive sampling method described in SectionV. Thisapproach is based on the assumption that the shadow surface
has uniform reflectance properties over the entire surface.

The last case is where the BRDF is not given, and the shadow
surface does not have uniform reflectance properties, e.g., surfaces with some textures. Even in this case, we are still able
to estimate an illumination distribution of a scene from shadows if it is conceivable that the shadow surface is a Lambertiansurface.
The question we have to consider here is how to cancel the
additional unknown number Kd in (6). An additional image ofthe scene taken without the occluding object, called the surface
image, is used for this purpose. The image irradiance of a surface image represents the surface color in the case where noneof the incoming light is occluded. Accordingly, in the case of
the surface image, the shadow coefficients S(θi , φi ) always become S(θi , φi ) = 1. Therefore, the image irradiance P  (θe , φe )of the surface image is computed from (6) asP  (θe , φe ) = Kdn
L(θj , φj )cosθj ωj

(10)

j=1

From (6) and (10), the unknown Kd is canceled.
Kd ni=1 L(θi , φi )cosθi S(θi , φi )ωi
P (θe , φe )=P  (θe , φe )Kd nj=1 L(θj , φj )cosθj ωj
n


=

i=1

L(θi , φi )cosθi S(θi , φi )ωiL(θj , φj )cosθj ωj
j=1

n

(11)
Finally, we obtain a linear equation for each image pixel
L(θi ,φi )where n L(θare unknowns, cosθi S(θi , φi )ωi are,φ )cosθ ω
j=1

j

j

j

j

e ,φe )
computable coefficients, and PP(θis a right-hand side quan(θe ,φe )tity. Again, if we use a sufficiently large number of pixels for theIEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. Y, NO. Y, MON 2002

5

Fig. 6. Input images : (a) surface image (b) shadow image

Fig. 8. Error Analysis: unknown reflectance property

the occluding object in the shadow image. This shows that theestimated illumination distribution gives a good representationof the characteristics of the real scene.
Fig. 8 numerically shows the improvement of the accuracy byincreasing the number of samplings. The estimated illumination
distribution was evaluated using a new occluding object with adifferent shape. From the plot in the figure, we can clearly seethat the accuracy improves as we use more sampling directionsof the illumination distribution. It is likely that the flatteningof the error curve before convergence to the exact solutions iscaused by the same factors described in Section IV-A.3.
Fig. 7. Synthesized images: unknown reflectance property

estimation, we are able to solve the set of linear equations for ai ,φi )
solution set of unknown n L(θ(i = 1, 2, · · · ·, n).
L(θ ,φ )cosθ
j=1

j

j

j

We should point out that the estimated radiance from theseequations is a ratio of the illumination radiance in one direction L(θi , φi ) to scene irradiance at the surface pointnj=1 L(θj , φj )cosθj ωj . As a result, we cannot relate the estimated radiance over the color bands unless the ratio of the scene
irradiance among color bands is provided. It is, however, not so
difficult to obtain this ratio. For instance, if there is a surface
with a white color in the scene, the recorded color of the surface directly indicates the ratio of the scene irradiance amongcolor bands. Otherwise, an assumption regarding total radianceis required.
B.3 Experimental Results for Non-Uniform Surface Material
The input images used in this experiment are shown in Fig. 6.
The illumination distribution of the scene was estimated using
the image irradiance of both the shadow image and the surface
image, and a synthetic occluding object was superimposed onto
the surface image using the estimated illumination distribution.
Synthesized results are shown in Fig. 7. Again, in the case of
520 nodes, the shadows in the resulting image resemble those of

V. A DAPTIVE E STIMATION OF I LLUMINATIOND ISTRIBUTION WITH U NKNOWN R EFLECTANCEP ROPERTIES OF S HADOW S URFACE
This section further concerns the following two issues thatfurther generalize the proposed approach. The first issue is how
to provide reflectance properties of a surface inside shadows incases where they are not given a priori, and the shadow surface
is not conceivable as a Lambertian surface.
The second issue is how to efficiently estimate an illumination distribution of a scene without losing the accuracy of theestimated illumination distribution. In Section IV, we have estimated an illumination distribution of a real scene by sampling
the distribution at an equal solid angle given as node directionsof a geodesic dome. For more efficiently estimating the illumination distribution of a scene using fewer sampling directionsof illumination radiance, we propose to increase sampling directions adaptively based on the estimation at the previous iteration, rather than by using a uniform discretization of the overall
illumination distribution in this section.
A. Basic Steps of the Proposed Approach
To take both diffuse and specular reflections of the shadow
surface into consideration, a simplified Torrance-Sparrow reflection model (9) described in Section IV-A.2 is reused. Basedon this reflection model, both the illumination distribution of theIEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. Y, NO. Y, MON 2002

scene and the reflectance properties of the shadow surface are
estimated from image brightness inside shadows as described in
the following steps.
1. Initialize the reflectance parameters of the shadow surface.
Typically, we assume the shadow surface to be Lambertian,and the diffuse parameter Kd is set to be the pixel value ofthe brightest point on the shadow surface. The specularparameters are set to be zero (Ks = 0, σ = 0). 5
2. Select image pixels whose brightness is used for estimating both the illumination distribution of a scene and thereflectance properties of the shadow surface. This selection is done by examining the coefficients ai in (7) in themanner described in the Appendix.
3. Estimate illumination radiance L(θi , φi ). Using the reflectance parameters (Kd , Ks , σ) and image brightness inside shadows in the shadow image, the radiance distribution L(θi , φi ) is estimated as described in Section IV-A.2.
4. Estimate the reflectance parameters of the shadow surface(Kd , Ks , σ) from the obtained radiance distribution of thescene L(θi , φi ) by using an optimization technique. (Section V-B)
5. Proceed to the next step if there is no significant change inthe estimated values L(θi , φi ), Kd , Ks , and σ. Otherwise,
go back to Step 3. By estimating both the radiance distribution of the scene and the reflectance parameters of the
shadow surface iteratively, we can obtain the best estimation of those values for a given set of sampling directionsof the illumination radiance distribution of the scene.
6. Terminate the estimation process if the obtained illumination radiance distribution approximates the real radiancedistribution with sufficient accuracy. Otherwise, proceed to
the next step.
7. Increase the sampling directions of the illumination distribution adaptively based on the obtained illumination radiance distribution L(θi , φi ). (Section V-C)
Then go back to Step 2.
In the following sections, each step of the proposed approach
will be explained in more detail.
B. Estimation of Reflectance Parameters of Shadow Surface
based on Radiance Distribution
In this section, we describe how to estimate the reflectanceparameters of the shadow surface (Kd , Ks , σ) by using the estimated radiance distribution of the scene L(θi , φi ).
Unlike the estimation of the radiance distribution of the scene
L(θi , φi ), which can be done by solving a set of linear equations, we estimate the reflectance parameters of the shadow surface by minimizing the sum of the squared difference between
the observed pixel intensities in the shadow image and the pixel
values computed for the corresponding surface points. Hence,
the function to be minimized is defined asf=m(Pj  − Pj )2

6

where Pj  is the observed pixel intensity in shadows cast by theoccluding object, Pj is the pixel value of the corresponding surface points computed by using the given radiance distribution of
the scene L(θi , φi ) in (9), m is the number of pixels used forminimization. In our method, the error function in (12) is minimized with respect to the reflectance parameters Kd , Ks , andσ by the Powell method to obtain the best estimation of thosereflectance parameters [25]. As has been noted, this approachis based on the assumption that the shadow surface has uniform
reflectance properties over the entire surface.
C. Adaptive Sampling of Radiance Distribution
If the estimated radiance distribution for a set of sampling directions does not approximate the illumination distribution of
the scene with sufficient accuracy, we increase the sampling directions adaptively based on the current estimation of the illumination radiance distribution.
Radiance distribution changes very rapidly around a direct
light source such as a fluorescent light. Therefore, the radiancedistribution has to be approximated by using a large numberof samplings so that the rapid change of radiance distribution
around the direct light source is captured. Also, to correctly
reproduce soft shadows cast by extended light sources, radiance distribution inside a direct light source has to be sampled
densely.
On the other hand, coarse sampling of radiance distribution isenough for an indirect light source such as a wall whose amountof radiance remains small. As a result, the number of sampling
directions required for accurately estimating an illumination distribution of a real scene becomes exceedingly large.
To overcome this problem, we increase sampling directions
adaptively based on the estimation at the previous iteration,rather than by using a uniform discretization of the overall illumination distribution. In particular, we increase sampling directions around and within direct light sources. 6
Based on the estimated radiance distribution L(θi , φi ) for the
sampling directions at the previous step, additional sampling directions are determined as follows.
Suppose three sampling directions with radiance values
L1 , L2 , and L3 are placed to form a triangle M1 as illustrated
in Fig. 9. To determine whether a new sampling direction needs
to be added between L1 and L2 , we consider the following costfunction.

(12)

j=1
5 Note that the initial value of K is not so important since there is a scalingdfactor between the reflectance parameters and illumination radiance values inany case. To fix the scaling factor, we need to perform photometric calibrationof our imaging system with a calibration target whose reflectance is given apriori.

Fig. 9. Subdivision of sampling directions

6 A similar sampling method has been employed in radiosity computation toefficiently simulate the brightness distribution of a room [4].

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. Y, NO. Y, MON 2002

7

U (L1 , L2 ) = dif f (L1 , L2 ) + αmin(L1 , L2 )angle(L1 , L2 )(13)
where dif f (L1 , L2 ) is the radiance difference between L1 andL2 , min(L1 , L2 ) gives the smaller radiance of L1 and L2 ,angle(L1 , L2 ) is the angle between directions to L1 and L2 ,and α is the manually specified parameter which determines therelative weights of those two factors. The first term is required
to capture the rapid change of radiance distribution around direct light sources, while the second term leads to fine samplingof the radiance distribution inside direct light sources. The additional term angle(L1, L2 ) is used for avoiding unnecessarily
dense sampling inside direct light sources. In our experiments,
α is set to 0.5.
If the cost U is large, a new sampling direction is added between L1 and L2 . In our experiments, we computed the costfunction values U for all pairs of neighboring sampling directions, then added additional sampling directions for the first 50%
of all the pairs in the order of the cost function values U .

Fig. 10. Input image : (a) shadow image taken of an indoor scene (b) theregion which synthesized images with the estimated radiance distribution andreflectance parameters are superimposed.

D. Experimental Results for Adaptive Sampling Method
We have tested the proposed adaptive sampling method by using real images taken in both indoor and outdoor environments.
In the following experiment, the adaptive sampling technique
was applied to a uniform unknown reflectance properties case.
The shadow image shown in Fig. 10 was used here as an input
of a indoor scene.
Starting from a small number of uniform sampling directionsof the illumination distribution, the estimation of the radiancedistribution of the scene was iteratively refined. At the sametime, the reflectance parameters (Kd , Ks , and σ) of the shadow
surface were estimated as explained in Section V-B. Then an
appearance of the shadow surface around the occluding object
was synthesized using the estimated radiance distribution of thescene and the estimated reflectance parameters of the shadow
surface.
The region inside the red rectangle in Fig. 10 (b) was replacedwith the synthesized appearances in the left column in Fig. 11.
The number of sampling directions of the radiance distribution
used for the estimation is shown under the resulting images.
Here, the resulting shadows of the synthetic occluding object resemble more closely those of the occluding object in the shadow
image as we increase the number of sampling directions basedon the proposed adaptive sampling technique. Finally, in thecase of 140 sampling directions, the synthesized shadows of theoccluding object blend into the input shadow image well, andfew distinct boundaries in the shadows are seen in the resulting
composite image.
To see how well the adaptive sampling of radiance distribution works in this example, we took an omni-directional imageof the office scene as a ground truth. The middle column ofFig. 11 shows the omni-directional image of the scene takenby placing a camera with a fisheye lens looking upward on
the shadow surface in Fig. 10 (a). The omni-directional image shows both direct light sources, i.e., the fluorescent lamps
in our office, and indirect light sources such as the ceiling and

Fig. 11. Adaptive refinement of illumination distribution estimation: (a) synthesized images with the estimated radiance distribution and reflectance parameters(b) adaptive refinement of sampling directions with a ground truth of an omnidirectional image of the scene (c) the estimated radiance values visualized forcomparison with the ground truth

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. Y, NO. Y, MON 2002

8

Fig. 13. Input image : (a) shadow image taken in an outdoor scene (b) theregion where synthesized images with the estimated radiance distribution andreflectance parameters are superimposed in Figure 14

Fig. 12. Error Analysis

walls. The right column of Fig. 11 shows the estimated radiancevalues visualized for comparison with the ground truth. In thoseimages in Fig. 11 (b) and (c), we can see that sampling directionsof the radiance distribution were nicely added only around the
direct light sources at each step by the proposed adaptive sampling framework, starting from the coarse sampling directions at
the top row.
Fig. 12 numerically shows the improvement of the accuracy
by adaptive refinement of sampling directions and the estimationof reflectance properties of the shadow surface. From the plot in
the figure, we can clearly see that the accuracy improves rapidly
as we adaptively increase the sampling directions of the radiancedistribution.
To confirm the merit of the adaptive sampling framework, we
also estimated the illumination radiance distribution with uniform sampling. In that case, even 300 uniformly sampled directions could not achieve the same level of accuracy as the estimation result obtained by 80 sampling directions with the proposed
framework.
Fig. 13 (a) shows another example image taken outside the entrance lobby of our building in the late afternoon. In this image,
we used the rectangular pole with two colors as an occluding
object casting shadows. In the same way as the previous example, an appearance of the shadow surface around the occluding
object, illustrated with a red rectangle in Fig. 13 (b), was synthesized by using the estimated radiance distribution of the scene
and the estimated reflectance parameters of the shadow surface.
Fig. 14 shows the resulting images obtained by the use of our
method. Although the grid pattern on the shadow surface ismissing in those synthesized images due to the assumption of
uniform reflectance on the shadow image, the appearance of the
shadow around the occluding objects is virtually indistinguishable in the case of 140 sampling directions. This shows that theestimated illumination distribution well represents the characteristics of the real scene.

Fig. 14. Adaptive refinement of illumination distribution estimation: synthesized images with the estimated radiance distribution and reflectance parameters

VI. D ISCUSSION
In the field of computer graphics, there have also been several methods proposed for recovering an illumination distribution of a real scene from appearance changes observed on object surfaces in the scene. Most of them have been conducted infully computer-generated environments where a designer specified the geometry and reflectance of whole scenes and the locations of all light sources [31], [14]; those methods were mainly
developed for assisting designers in achieving desirable effects
such as highlights and shadows in the synthesized image.
Some researchers extended similar approaches for real images and estimated radiance of real light sources in the scene.
Fournier et al. [6] estimated radiance of the light sources withusers’ defined fix locations on the assumption that all surfaces
were Lambertian surfaces. Later, Marschner and Greenberg [20]introduced to approximate the entire illumination with a set ofbasis lights located in the scene. Although this method has anadvantage over the previous methods of not requiring knowl-

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. Y, NO. Y, MON 2002edge about the light locations of the scene, the estimation depends entirely on the appearance changes observed on an object
surface assumed to be Lambertian, and therefore some restrictions are imposed on the shape of the object, e.g., the object musthave a large amount of curvature. Recently, Ramamoorthi and
Hanrahan [26] introduced a signal-processing framework that
described the reflected light field as a convolution of the lighting and BRDF, and showed under which condition lighting and
BRDF recovery could be done robustly. Simultaneously, Basriand Jacobs [2] reported that the images of a convex Lambertian object obtained under a wide variety of lighting conditions
could be approximated with a low-dimensional linear subspaceby using a similar signal-processing approach.
We are currently investigating the analysis of the problem ofillumination estimation from shadows in their signal-processing
framework. It was found through our preliminary experimentsthat high frequency components of the appearance of an object
surface could retain significant energy by taking the shadow coefficients on the object surface as well as its BRDF into account.
This indicates that the use of shadows for the illumination estimation has the significant advantage of providing more clues tothe high frequency components of illumination distribution of ascene.
VII. C ONCLUSIONSIn this study, we have presented a method for recovering an
illumination distribution of a scene from image brightness observed on a real object surface. In a natural illumination condition, it is hard to recover an illumination distribution from image
brightness observed on a real object surface. The main reason
for this difficulty is that there is usually not adequate variation in
the image brightness observed on the surface to reflect the subtle
characteristics of the entire illumination.
One of the main contributions of our work is to demonstrate
the effectiveness of using occluding information of incoming
light in estimating the illumination distribution of a scene. Analyzing the relationships between the image brightness and theocclusions of incoming light enabled us to reliably estimate theillumination distribution of a scene even in a complex illumination environment. In addition, the question of how to provide
the reflectance properties of a surface to be used for the estimation still remains from the previously proposed methods. Since
it is a common situation that reflectance properties of a surface
are not known, solutions to this problem are required. Anothercontribution of our work is that we have combined the illumination analysis with an estimation of the reflectance propertiesof a surface in the scene. This makes the method applicable tothe case where reflectance properties of a surface are not known,
and it enlarges the variety of images to which the method can beapplied.
We also introduced an adaptive sampling of the illumination
distribution of a scene as a solution to the question of how we
could efficiently estimate the entire illumination with a smaller
number of sampling directions of the entire distribution in thispaper. Using the proposed adaptive sampling framework, wewere able to avoid unnecessarily dense sampling of the illumination and to estimate overall illumination more efficiently byusing fewer sampling directions.

9

While the effectiveness of the proposed method in estimating illumination distributions of usual scenes was demonstrated
in this paper, the estimation was based on the assumptions thatlight sources in the scene were sufficiently distant from the objects, and that there was no severe interreflection between ashadow region and an occluding object. The future directionsof this study include extending the method for: (1) considering
the interreflection between a shadow region and an occluding
object; and (2) taking distances from the occluding object tolight sources into account.
ACKNOWLEDGMENTS
This research was supported in part by the Ministry of Education, Science, Sports and Culture grant-in-Aid for Creative Basis
Research 09NP1401 and Research Fellowships of the Japan Society for the Promotion of Science for Young Scientists. The
authors are grateful to the anonymous reviewers whose suggestions helped to improve the paper.
A PPENDIX
The proposed method estimates radiance value of the lightsources based on the variations in the computed coefficients aiin (7); we obtain n coefficients ai (i = 1, 2, · · · ·, n) per image pixel
where n is the number of the light sources. For reliable estimation, it is essential that we select image pixels from a shadow
image to maximize variation in the combinations of ai .
In our approach, the shadow surface is first partitioned into
clusters based on combinations of the coefficients ai . In otherwords, pixels that have the same combination of ai are clusteredinto the same group. Fig. 15 shows several examples of partitioning based on the coefficients ai with pseudo colors. Hereeach color represents an individual class with a different combination of coefficients of the light sources, and the block region
corresponds to the occluding objects.
After the shadow surface are partitioned into clusters, one
pixel is selected from each of those clusters. From this clustering process, we are able to avoid selecting redundant pixels
that provide the same information about the illumination of thescene. This results in significant speedup of the entire estimation
process as well.

Fig. 15. Clustering results

R EFERENCES[1][2]

R. Baribeau, M. Rioux, and G. Godin, “Color Reflectance Modeling Usinga Polychromatic Laser Range Sensor,” IEEE Trans. PAMI, vol. 14, no. 2,
1992, pp. 263-269.
R. Basri and D. Jacobs, “Lambertian Reflectance and Linear Subspaces,”
Proc. IEEE Intl. Conf. Computer Vision 01, 2001, pp.383-389.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. Y, NO. Y, MON 2002

[3]

J. Bouguet and P. Perona, “3D Photography on Your Desk,” Proc. IEEE
Intl. Conf. Computer Vision 98, 1998, pp.43-50.
[4] M. F. Cohen, D. P. Greenberg, D. S. Immel, and P. J. Brock, “An EfficientRadiosity Approach for Realistic Image Synthesis,” IEEE CG & A, Vol. 6,
No.3, Mar. 1986, pp.26-35.
[5] D. A. Forsyth and J. Ponce, Computer Vision A Modern Approach, Prentice Hall, Upper Saddle River, NJ., 2002, pp. 70.
[6] A. Fournier, A. Gunawan and C. Romanzin, “Common Illumination between Real and Computer Generated Scenes,”Proc. Graphics Interface 93,
1993, pp.254-262.
[7] G. E. Healey, S. A. Shafer, and L. B. Wolff, Physics-Based Vision Principles and Practice, Color, Jones and Bartlett Publishers, Boston, MA.,
1992.
[8] B. K. P. Horn,“Understanding Image Intensities,” Artificial Intelligence,Vol. 8, No.2, 1977, pp.201-231.
[9] B. K. P. Horn, Robot Vision, The MIT Press, Cambridge, MA., 1986.
[10] B. K. P. Horn, “Obtaining Shape from Shading Information,” The psychology of Computer Vision, McGraw-Hill Book Co., New York, N.Y., 1975.
[11] B. K. P. Horn and M. J. Brooks, “The Variational Approach to Shape from
Shading,” Computer Vision, Graphics, and Image Processing, Vol.33.
No.2, 1986, pp.174-208.
[12] K. Ikeuchi and B. K. P. Horn, “Numerical Shape from Shading andOccluding Boundaries,” Artificial Intelligence, Vol. 17, No.1-3, 1981,
pp.141-184.
[13] K. Ikeuchi and K. Sato, “Determining Reflectance using Range andBrightness Images,” Proc. IEEE Intl. Conf. Computer Vision 90, 1990,
pp.12-20.
[14] J. Kawai, J. Painter, and M. Cohen, “Radioptimization - Boal Bawsed Rendering”Proc. ACM SIGGRAPH 93, 1993, pp. 147-154.
[15] G. Kay and T. Caelli, “Estimating the Parameters of an Illumination Modelusing Photometric Stereo,”Graphical Models and Image Processing, vol.
57, no. 5, 1995, pp. 365-388.
[16] J. R. Kender and E. M. Smith, “Shape from Darkness: Deriving Surface
Information from Dynamic Shadows,” Proc. IEEE Intl. Conf. Computer
Vision 87, 1987, pp.539-546.
[17] T. Kim, Y. Seo, and K. Hong, “Improving AR using Shadows Arisingfrom Natural Illumination Distribution in Video Sequence,”Proc. IEEE
Intl. Conf. Computer Vision 01, July 2001, pp. 329-334.
[18] J. Lu and J. Little, “Reflectance Function Estimation and Shape Recovery
from Image Sequence of a Rotating Object,”Proc. IEEE Intl. Conf. Computer Vision 95, 1995, pp. 80-86.
[19] A. K. Mackworth, On the Interpretation of Drawings as ThreeDimensional Scenes, doctorial dissertation, University of Sussex, 1974.
[20] S. R. Marschner and D. P. Greenberg, “Inverse Lighting for Photography,”Proc. IS&T/SID Fifth Color Imaging Conference, 1997, pp.262-265.
[21] P. Moon and D. E. Spencer, The Photic Field, The MIT Press, Cambridge,
MA., 1981.
[22] S. K. Nayar, K. Ikeuchi, and T. Kanade, “Surface Reflection: physical andgeometrical perspectives,” IEEE Trans. PAMI, vol. 13, no. 7, 1991, pp.
611-634.
[23] K. Nishino, Z. Zhang, and K. Ikeuchi, “Determining Reflectance Parameters and Illumination Distribution from Sparse Set of Images for Viewdependent Image Synthesis,” Proc. IEEE Intl. Conf. Computer Vision 01,
Jul. 2001, pp. 599-606.
[24] A. P. Pentland, “Linear Shape From Shading,” Intl. J. Computer Vision,
Vol.4, No.2, 1990, pp.153-162.
[25] W. H. Press, B. P. Flannery, S. A. Teukolsky, W. T. Vetterling, Numerical Recipes in C: The Art of Scientific Computing, Cambridge University
Press, Cambridge, 1988.
[26] R. Ramamoorthi and P. Hanrahan, “A Signal-Procession Framework for
Inverse Rendering”, Proc. ACM SIGGRAPH 01, Aug. 2001, pp.117-128.
[27] I. Sato, Y. Sato, and K. Ikeuchi, “Illumination Distribution from Shadows,”
Proc. IEEE Conf. Computer Vision and Pattern Recognition 99, Jun. 1999,
pp. 306-312.
[28] I. Sato, Y. Sato, and K. Ikeuchi, “Illumination Distribution from Brightness in Shadows: adaptive estimation of illumination distribution with unknown reflectance properties in shadow regions,” Proc. IEEE Intl. Conf.
Computer Vision 99, Sep. 1999, pp. 875-882.
[29] I. Sato, Y. Sato, and K. Ikeuchi, “Acquiring a radiance distribution to superimpose virtual objects onto a real scene”, IEEE Trans. TVCG, Vol. 5,
No. 1, 1999, pp. 1-12.
[30] Y. Sato, M. D. Wheeler, and K. Ikeuchi, “Object shape and reflectancemodeling from observation,” Proc. ACM SIGGRAPH 97, 1997, pp. 379387.
[31] C. Schoeneman, J. Dorsey, B. Smits, J. Arvo, and D. Greenburg, “Painting
with Light,”Proc. ACM SIGGRAPH 93, 1993, pp. 143-146.
[32] S. A. Shafer and T. Kanade, “Using Shadows in Finding Surface Orienta-

10

tions,” Computer Vision, Graphics, and Image Processing, Vol.22, 1983,
No.1, pp. 145-176.
[33] S. Tominaga and N. Tanaka, “Estimating Reflectance Parameters from aSingle Color Image,” IEEE Computer Graphics & Applications, vol.20,
No. 5, 2000, pp.58-66.
[34] K. E. Torrance and E. M. Sparrow, “Theory for off-specular reflection fromroughened surface,” J. Optical Society of America, vol.57, 1967, pp.11051114.
[35] Y. Yu, P. Devebec, J. Malik, and T. Hawkins, “Inverse Global Illumination:
Recovering reflectance Models of Real Scenes from Photographs”, Proc.
ACM SIGGRAPH 99, Aug. 1999, pp.215-224.
[36] Y. Zang and Y. Yang, “Illuminant Direction Determination for Multiple
Light Sources,” Proc. IEEE Conf. Computer Vision and Pattern Recognition 00, Jun. 2000, pp. 269-276.
[37] 3D Construction Company, http://www.3dcomstruction.com

Imari Sato is a PhD student in Graduate School ofInterdisciplinary Information Studies at the Universityof Tokyo, Tokyo, Japan. She received the BS degree
in policy management from Keio University, Kanagawa, Japan, in 1994 and the MS degree in interdisciplinary information studies from the University ofTokyo in 2002. Previously, she was a visiting scholar
at the Robotics Institute at Carnegie Mellon University, Pittsburgh, Pennsylvania, USA. Her research interests include physics-based vision and image-basedmodeling. She is a research fellow of the Japan Society for the Promotion of Science.

Yoichi Sato received the BS degree in mechanicalengineering from the University of Tokyo, Tokyo,
Japan, in 1990. He received the MS and PhD degrees in robotics from the School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA in 1993 and 1997, respectively. In
1997, he joined the Institute of Industrial Science at
the University of Tokyo, where he is currently an associate professor. His primary research interests arein the fields of computer vision (physics-based vision,
image-based modeling), human-computer interaction(perceptual user interface), and augmented reality. He is a member of the IEEE.

Katsushi Ikeuchi received the BEng degree in mechanical engineering from Kyoto University, Kyoto,
Japan, in 1973, and the PhD degree in information engineering from the University of Tokyo, Tokyo, Japan,
in 1978. He is a professor at the Institute of Industrial
Science, the University of Tokyo. After working at
the Artificial Intelligence Laboratory at Massachusetts
Institute of Technology, the Electrotechnical Laboratory of Ministry of International Trade and Industries,
and the School of Computer Science, Carnegie Mellon University, he joined the University of Tokyo in
1996. He has received various research awards, including the David Marr Prizein computational vision in 1990, and IEEE R&A K-S Fu Memorial Best Transaction Paper award in 1998. In addition, in 1992, his paper, ”Numerical Shapefrom Shading and Occluding Boundaries,” was selected as one of the most influential papers to have appeared in the Artificial Intelligence Journal within thepast ten years. He is a fellow of the IEEE.