Hemispherical Confocal Imaging
using Turtleback Reflector
Yasuhiro Mukaigawa1,2 , Seiichi Tagawa1 , Jaewon Kim2 ,Ramesh Raskar2 , Yasuyuki Matsushita3 , and Yasushi Yagi1
1

Osaka University
MIT Media Lab
3
Microsoft Research Asia
2

Abstract. We propose a new imaging method called hemispherical confocal imaging to clearly visualize a particular depth in a 3-D scene. The
key optical component is a turtleback reflector which is a specially designed polyhedral mirror. By combining the turtleback reflector with acoaxial pair of a camera and a projector, many virtual cameras and projectors are produced on a hemisphere with uniform density to synthesize
a hemispherical aperture. In such an optical device, high frequency illumination can be focused at a particular depth in the scene to visualize
only the depth with descattering. Then, the observed views are factorized
into masking, attenuation, and texture terms to enhance visualization
when obstacles are present. Experiments using a prototype system showthat only the particular depth is effectively illuminated and hazes by
scattering and attenuation can be recovered even when obstacles exist.

1

Introduction

Significant effort has been made to obtain cross-sectional views of a 3-D scene.
Real scenes often include obstacles such as scattering materials or opaque occluders. To clearly visualize cross-sectional views as if the scene is cut at a plane,
only a particular depth has to be illuminated and haze due to scattering andattenuation should be recovered.
The simplest way to observe a particular depth is to use a large aperturelens. The large aperture makes the DOF (depth of field) shallow, and the region
outside the DOF is blurred. The synthetic aperture method [24] mimics a large
virtual aperture by combining many small apertures. However, obstacles are stillbright and visible, while they are blurred. The confocal imaging [15] simultaneously scans two confocal pinholes over a particular depth. Since both illuminationand observation are focused, clear cross-sectional views are obtained. While stillvisible, obstacles are darkened and blurred. Moreover, scanning requires longmeasuring time.
Recently, Levoy et al. [11] proposed a new imaging technique which combines
synthetic aperture with the confocal imaging. Since this technique is based onlight field analysis, only a particular depth can be illuminated without scanning.
However, the synthesized aperture size is relatively small because rectangular

2

Y.Mukaigawa, S.Tagawa, J.Kim, R.Raskar, Y.Matsushita, and Y.Yagi

mirrors are aligned as a 2D array. Moreover, unwanted effects such as scatteringand attenuation still remain.
In this paper, we propose a novel imaging method called hemispherical confocal imaging. To improve the imaging performance of the synthetic aperture
confocal imaging [11], we designed turtle reflector which is a polyhedral mirror
to approximate a hemispherical aperture with 180 degree of FOV. We introduce
focused high frequency illumination using the turtleback reflector with a projector. This method can eliminate scattering on the focused depth, and makeunfocused depth almost invisible. Moreover, we introduce factorization of observed views to eliminate attenuation.
Contribution– By utilizing the new optical device, unfocused depth becomes almost invisible
and scattering is eliminated. Moreover, the measurement is very fast, sinceno scanning is required.
– We have designed the turtleback reflector which is a polyhedral mirror circumscribed in an ellipsoid. The combination of the turtleback reflector, aprojector, and a camera can synthesize a hemispherical wide aperture. The
optical device can also be used for measuring the complete 8-D reflectancefield on the hemisphere.
– A new imaging technique of the focused high frequency illumination is introduced. This technique enables us to separate direct and global components
not on the surface but in the 3-D space because any projection pattern canbe focused at the particular depth in the 3-D scene.

2

Related work

2.1

Reflectance field measurement

The optical device proposed in this paper can be regarded as an 8-D reflectancefield measuring device. A 4-D slice of the 8-D reflectance field under a static
illumination can be recorded by scanning a camera [12] or installing multiple
cameras [24]. Alternatively, a high-resolution camera is combined with a microlens array [1], a micro mirror array [22], or masks [25]. To vary illumination,
Debevec et al. [4] rotated a light source, and Sen et al. [20] used a projector asa light source. Masselus et al. [13] rotated a projector, and Matusik et al. [14]rotated both a light source and a camera to measure 6-D reflectance field. Müller
et al.[17] used 151 cameras with flashes.
In principle, complete 8-D reflectance field can be measured by densely installing many projectors and cameras on a hemisphere. However, it is difficult torealize such a system due to the cost and physical interference between devices.
While rotating projector and camera solves these problems, capture process isimpractically long. Recently, Garg et al. [6] and Levoy et al. [11] used multiple
planar mirrors and Cossairt et al. [2] used a lens array to measure a part of 8-D

Hemispherical Confocal Imaging using Turtleback Reflector

3

reflectance field, but the direction of the illumination and observation is limited
to a narrow angle.
On the other hand, only our system can measure complete 8-D LF on thehemisphere covering the scene. Since our system utilizes the geometric property of an ellipsoid, many virtual cameras and projectors can be produced on ahemisphere with uniform density.
2.2

BRDF measurement using mirrors

For measuring bidirectional reflectance distribution function (BRDF), mirrors
are often used to replace mechanical motion. Reflected lights can be effectively
measured from all directions using a hemispherical mirror [26] or a paraboloidal
mirror [3]. Recently, a wide variety of mirrors such as a cylindrical mirror [10],several plane mirrors [9], an ellipsoidal mirror [16], and a combination of aparaboloidal mirror and a specially-designed dome mirror [7] have been usedin conjunction with a projector and a camera.
Our turtleback reflector design was inspired by the BRDF measurement using
an ellipsoidal mirror [16]. They utilized a geometric property of a rotationallysymmetric ellipsoid that all rays from one focal point reflect off the ellipsoidal
mirror and reach the other focal point. On the other hand, we utilized a different
geometric property of an ellipsoid that the total length from one focal point
to the other focal point through any surface points is constant. By utilizing
this characteristic, we can produce virtual cameras and projectors at a constantdistance from the target just as they are on a hemisphere. Since our purpose is
not BRDF measurement but visualization of cross-sectional views, we designeda polyhedral mirror circumscribed in an ellipsoid.
2.3

Descattering

Incident lights to murky liquid or translucent media scatter, and the appearance becomes blurred. To obtain clear views, descattering methods have beendeveloped. Treibitz and Schechner [21] used polarizer under water. Assumingthat only single scattering is observed in optically thin media, Narasimhan etal. [18] estimated 3-D shape with descattering and Gu et al. [8] estimated 3-D
distribution of inhomogeneous scattering media.
Recently, Fuchs et al. [5] combined confocal imaging with descattering which
utilize the fact that scattering components have low frequency. The principleof our hemispherical confocal imaging is similar to this approach. However, we
combine two ideas of the focused illumination proposed by Levoy et al. [11] andthe high frequency illumination proposed by Nayar et al. [19]. Our approachboth shortens the measuring time and eliminates scattering.

3

Hemispherical confocal imaging

Let us assume that a 3-D scene is illuminated by a light source and observed bya camera as shown in Fig.1. Even if the camera is focused on a particular depth

4

Y.Mukaigawa, S.Tagawa, J.Kim, R.Raskar, Y.Matsushita, and Y.Yagi

scattering

occlusion
attenuation
absorption
obstacles

unfocuseddepth
focused depth

Fig. 1. Illumination and reflection in a 3-D scene. It is difficult to observe a particulardepth due to scattering and attenuation.
Table 1. Comparison of several imaging methods.

Synthetic aperture
Confocal imaging
Synthetic aperture confocal imaging[11]Confocal imaging with descattering [5]Our hemispherical confocal imaging

unfocused depth scanning scattering
brightunnecessary remaining
darkennecessary remaining
unilluminated unnecessary remaining
darkennecessary
reducedunilluminated unnecessary reducedin the scene, the captured image includes reflections from the entire scene. To
observe the particular depth, only the depth should be illuminated. This meansthat both the illumination and observation should have a shallow DOF.
Even if we succeed in illuminating only the particular depth, clear views cannot be observed. The major reasons are scattering and attenuation. The scattering is caused by multi-bounce reflections in translucent media. By the scattering,
the views become blurred. On the other hand, the attenuation is caused by occlusion due to obstacles or absorption due to low transparent media. By theattenuation, the illumination becomes nonuniform and the reflections are partially darkened. The following four functions are required to obtain clear viewsof a particular depth in a 3-D scene.
(a) The DOF should be as shallow as possible.
(b) Only the particular depth should be illuminated.
(c) Scattering should be eliminated.
(d) Attenuation should be eliminated.
To satisfy these requirements, we propose the hemispherical confocal imaging
consisting of (1) specially designed turtleback reflector, and (2) focused highfrequency illumination.
The turtleback reflector with coaxial camera and projector synthesizes ahemispherical aperture for both illumination and observation to solve (a). The
focused high frequency illumination eliminates reflections from unfocused depth
and global reflection to solve (b) and (c). Then, we factorized the observed views
into masking, attenuation, and texture terms to solve (d). The merits and de-Hemispherical Confocal Imaging using Turtleback Reflector

5

Relighting with 4D incident light fields (Masselus 2003)
Hemispherical confocal imaging (Ours 2010)
Synthetic aperture confocal imaging (Levoy 2004)
Symmetric photography (Garg 2006)
Combining confocal imaging and descattering (Fuchs 2008)
High frequency illumination (Nayar 2006)
Dual photography (Sen2005)
Synthetic aperture

m
an
y

P
ro
je
ct
or

on
e
one

Cameramany

Fig. 2. The number of projectors and cameras of several imaging methods which use
projector(s) and camera(s) for reflection analysis or reflectance field measurement.
hemispherehemispherevirtual cameras
and projectors
tangent planevirtual cameras
and projectors

tangent plane

ellipsoid
real camera and projector

target object(a) design using single ellipsoid

two sets ofcamera and projector

target object
two ellipsoids(b) design using dual ellipsoids

Fig. 3. Principle of the proposed optical device. Virtual projector and cameras are
distributed on a hemisphere with uniform density by using the turtleback reflector.

merits and the number of projector and cameras of several imaging methods are
summarized in Table 1 and Fig.2, respectively.

4

Optical design of turtleback reflector

4.1

Projectors and cameras on a hemisphere

In our system, projectors are used as light sources to illuminate a particulardepth. If the aperture size of the projector is large enough, the projected pattern
is focused only within the shallow DOF and blurred elsewhere. Although it isdifficult to realize a large aperture using a single lens, the combination of anumber of small apertures can easily synthesize a large aperture by the synthetic
aperture technique[24].
To mimic an extremely large aperture, many projectors should be placedin every direction at a constant distance and uniform density. That is, ideal

6

Y.Mukaigawa, S.Tagawa, J.Kim, R.Raskar, Y.Matsushita, and Y.Yagi

75mm

100mm

(a) front view(b) bottom view

90mm(c) dimensions

Fig. 4. Design of the turtleback reflector.

locations of the projectors and cameras are on a hemisphere. If both the projectors and cameras are densely distributed on the hemisphere covering the target
scene, the projectors and cameras can synthesize a hemispherical aperture with180 degree of FOV for both illumination and observation.
4.2

Turtleback reflector for virtual projectors and cameras

It is difficult to place many projectors and cameras due to the cost and physicalconflict. Therefore, we distribute many virtual projectors and cameras using
mirrors. For this purpose we utilize the geometric property of an ellipsoid thatthe total length from one focal point to the other focal point through any surface
points on a rotationally symmetric ellipsoid is a constant. A target object isplaced at the focal point, and a projector and a camera are placed at the otherfocal point using beam splitter as illustrated in Fig.3(a). Planner mirrors areplaced in the tangent planes to the ellipsoid. This is equivalent to hemispherical
distribution of many virtual projectors and cameras with uniform density.
Actually, the design using single ellipsoid is difficult to construct because it
requires real projector and camera with 180 degree of FOV. Hence, the hemisphere is evenly divided into two parts and two slanted ellipsoids are placed sothat they share same focal point as shown in Fig.3(b). In this design using dual
ellipsoids, a projector and a camera with a normal FOV can be used.
4.3

Design of optical device

We designed a new turtleback reflector which is a polyhedral mirror circumscribed in an ellipsoid as shown in Fig.4. The virtual projectors and cameras
are placed at the nodes of a geodesic dome which is generated by dividing an
icosahedron two times. While the optical device is composed of two symmetricparts, we constructed one half as a prototype to confirm the ability.
Figure 5(a) shows the frame to fix the mirror patches made by stereolithography. Fifty mirror patches are attached to the frame. The turtlebackreflector is combined with a high-resolution camera (PointGrey, Grass-50S5C,
2448 × 2048) and a projector (KAIREN Projector X Pro920, 640 × 480).

Hemispherical Confocal Imaging using Turtleback Reflector

projector

target object

turtleback reflector(a) turtleback reflector

7

er
litt
sp
m
a
be

camera(b) total optical device

Fig. 5. Optical device for hemispherical confocal imaging.

5

Focused high frequency illumination

5.1

Illumination and reflection in a 3-D scene

To analyze the reflectance field in a 3-D scene, we need to know how lights
illuminate points in a scene, and how the reflections are observed. We divide the3-D scene into a set of small voxels. Let Lk be a set of rays which illuminate
the k-th voxel, and R(Lk ) be a set of reflected rays of Lk at the voxel. Since
the observed view of the entire scene by a camera is expressedby a sum of theP
reflected rays from all voxels, the view is presented by k R(Lk ).
Illuminations and reflections can be regarded as a sum of direct and global
components[19]. As shown in Fig.6 (a), the illumination of the k-th voxel can beGdecomposed into the direct illumination LD
k and the global one Lk . Similarly, theDreflection can also be decomposed into the direct reflection R (Lk ) and global
one RG (Lk ). That is,
R(Lk ) = RD (Lk ) + RG (Lk )G
and Lk = LD
k + Lk .

(1)

Hence, the observed view can be modeled as a sum of four components by
X
X
G
D
D
G
G
G
R(Lk ) =(RD (LD(2)k ) + R (Lk ) + R (Lk ) + R (Lk ))
k

5.2

k

Focused illumination by multiple projectors

To obtain clear views of a particular depth in a 3-D scene, only the depth should
be illuminated. Moreover, any global illuminations and global reflections shouldbe eliminated to reduce scattering in the media. That is, the first term RD (LDk )in Eq.(2) should be measured separately .

8

Y.Mukaigawa, S.Tagawa, J.Kim, R.Raskar, Y.Matsushita, and Y.Yagi

LkD
LkG

k-th voxel

RD(Lk)
RG(Lk)(a) four types of ray

:U
:F1
:F2

(b) focused illumination (c) classification of voxels

Fig. 6. Focused high frequency illumination. The high frequency patterns are focused
only on the particular depth. The projection is blurred out of the DOF.

By using our optical device, such special illumination and measurement canbe realized. Since the virtual projectors surrounding the target scene can synthesize a large aperture, the DOF becomes very shallow. We combine the focused
illumination technique proposed by Levoy et al.[11] and high frequency illumination technique proposed by Nayar et al.[19], and call the new techniquefocused high frequency illumination (FHFI for short). It is noted that our optical system enables the efficient combination of [11] and [19]. The former canproject arbitrary focused patterns at any plane in the scene. Out of the focused plane, projected patterns become blurred. The latter can separate direct
and global components on the 2-D surface. Our FHFI can separate direct andglobal components in the 3-D volume, because our optical device can synthesize
a hemispherical aperture with 180 degree of FOV.
For the FHFI, high frequency checker board patterns are projected from eachprojector. The position of the white and black pixels are aligned at the depth asshown in Fig.6 (b). This means that the high frequency illumination is focused
only at a particular depth. The voxels in the scene are classified into unfocusedvoxels U , and focused and illuminated voxels F 1, and focused but unilluminatedvoxels F 2 as shown in Fig.6 (c).
Compared to a white pattern, the average intensity of the high frequency
illumination is darken because the half of pixels are black. Table 2 shows therelative intensities of the four reflection components for each voxel type. The
global illumination to every voxel decreases by half. The direct illumination to
U also decreases by half because the projected patterns are blurred. The F 1
receives full direct illumination, P
while the F 2 receives no direct illumination.
By combining these differences, k∈F 1∪F 2 RD (LDk ) which presents only directcomponents from voxels at the focused depth can be separated.
Let IP be a captured image when voxels of F 1 are illuminated but voxels of
F 2 are not illuminated. Let IN be a captured image when the inverse pattern is
projected. Then, these images can expressed as X  G X  D

X 
LiG
Li
Li + LiGDIP =R Li +
+
R
+
R
, (3)
2
2
2
i∈F 1
i∈F 2
i∈U
 X  D
X  LiG  X 
LiG
Li + LiG
IN =R+
R LiD +
+
R
. (4)
2
2
2
i∈F 1

i∈F 2

i∈U

Hemispherical Confocal Imaging using Turtleback Reflector

9

Table 2. Relative intensities of four reflection components for each voxel type.
D
G
G
D
G
G
RD (LDk ) R (Lk ) R (Lk ) R (Lk )U (unfocused)
1/2
1/2
1/2
1/2
F1 (focused and illuminated)
1
1/2
1
1/2
F2 (focused and unilluminated)
0
1/2
0
1/2

By comparing two intensities at same position in IP and IN , we can make animage Imax which has larger intensities and an image Imin which has smaller
intensities. Since global component has only low frequency [5] [19],X
X
RG (LiD ) 'RG (LiD ).
(5)
i∈F 1

i∈F 2

Therefore,
Imax − Imin =X

RD (LiD ) +

i∈F 1

=

X

RD (LiD ) ± (

i∈F 2
D

D

R (Li ) +

i∈F 1

=

X

X

X

X
i∈F 1

D

RG (LiD ) −X

RG (LiD ))

i∈F 2

D

R (Li )i∈F 2

RD (LiD ).

(6)

i∈F 1∪F 2

This means that only the particular depth (F 1 ∪ F 2) can be directly illuminated without global illuminations, and only the direct reflections can bemeasured without global reflections. As shown in Table 1, our method does notilluminate unfocused depth. Since no scanning is necessary, the measurement
is fast. Moreover, scattering which is a major global component in translucent
media is eliminated.

6

Factorization of the observed views

Although only the focused depth is illuminated and scattering is eliminated by
the FHFI, the view of the focused depth is still unclear due to attenuation of theincident and reflective lights. This is due to occlusion and absorption as shown in
Fig.1. Occlusion casts sharp shadows because some lights are directly interrupted
by obstacles, while attenuation usually makes smooth change because lighting
powers are decreased by spatially distributed low transparent media.
Fortunately, the scene is observed by several virtual cameras. Even if somelights are attenuated, other cameras may observe the scene without attenuation.
Hence, we try to estimate the texture which is not affected by attenuation basedon the observation from multiple cameras. We assume that there are K virtual
cameras and each camera has N pixels. Let Oij be the intensity of the i-th pixelin the j-th camera. We model that the observed intensities are factorized as
Oij = Mij Aij Ti .

(7)

10

Y.Mukaigawa, S.Tagawa, J.Kim, R.Raskar, Y.Matsushita, and Y.Yagi

i

0

j
K

N

x

=
O : Observed views M={0,1}

ij

ij: Masking termx
A : Attenuation termij

T : Texture termiFig. 7. Concept of the factorization. The observed intensities are factorized into themasking, attenuation, and texture terms to reduce attenuation.

Here, Mij is the masking term which has a value of 0 or 1. If the light is occluded
by obstacles, the value becomes 0, otherwise it becomes 1. Aij is the attenuation
term which expresses light attenuation due to absorption. Ti is the texture term
which expresses the reflectance of the particular depth. It is noted that only
the texture term is independent to the viewing direction assuming Lambertianreflection. Figure 7 illustrates this relationship.
The flow of the factorization process is as follows
STEP-1: First, the masking term is decided. Since unfocused depths are
not illuminated by the FHFI, obstacles can be easily distinguishedusing a simple threshold. After decision of the masking term, thefollowing processes are done for pixels satisfying Mij = 1.
STEP-2: The initial attenuation term is decided as Aij = 1.
STEP-3: The texture term is calculated. Ideally, a unique reflectanceshould be estimated despite different camera j, but the observed intensities vary. This kind of problem is often seen in stereoscopy[23], so
we used a median filter in a similar fashion by Ti = Median(Oij /Aij ).
STEP-4: Update the attenuation term by Aij = Oij /Ti to satisfy
Eq.(7).
STEP-5: Smooth the attenuation term using a Gaussian function, because attenuation smoothly varies over the 3-D scene. Then go backto STEP-3 until the texture term does not change.
By this factorization process, the observed views are decomposed to three terms
and we can get clear texture of the particular depth without attenuation.

7

Experiments

7.1

Synthetic aperture using virtual cameras

First, we evaluated the ability of the synthetic aperture using the prototype
system. A textured paper is covered by obstacle of yellow dense mesh as shownin Fig.8(a). A white uniform pattern was projected onto the scene. Figure 8(b)shows the captured image by the real camera. This image includes fifty views
corresponding to fifty virtual cameras. Since all views are affected by the obstacle, it is difficult to see the texture of the sheet. Figure 8(c) shows the change ofHemispherical Confocal Imaging using Turtleback Reflector(a) target scene

11

1 view

2 views

3 views

4 views

20 views

30 views

40 views

50 views(b) captured image(c) change of the aperture size

Fig. 8. Result of synthetic aperture using our optical device.

(a)

(b)

(c)

Fig. 9. Descattering by the focused high frequency illumination. Lefts: views undernormal illumination. Rights: estimated direct components.

the appearance when the number of the virtual cameras increases to synthesize
a large aperture. Since our optical device can synthesize a half of the hemispherical aperture, the obstacle is completely blurred and the texture becomes clear
with increasing the number of virtual cameras.
7.2

Descattering by focused high frequency illumination

Next, we confirmed that the FHFI is effective for descattering in a 3-D volume.
We covered some textured papers with a white plastic sheet. The left imagesof Fig.9 (a)(b)(c) show views when a white uniform pattern was projected. The
appearances are blurred due to scattering. Checkered patterns in which whiteand black are replaced every three pixels are projected from the virtual projectors
so that these patterns are aligned on the paper. Totally, eighteen images were
captured by shifting the projecting pattern. The right images of Fig.9 (a)(b)(c)show the direct component when the high frequency patterns were projected. We
can see that scattering in the 3-D scene is effectively reduced and the appearances
become clear. While the descattering effect is not perfect, this is attributed to
the low resolution of the virtual projectors in our current prototype system.
7.3

Factorization of the observed views

We confirmed the ability to visualize a particular depth in a 3-D scene by combining the FHFI and the factorization. Figure 10(a) shows the scene that an

12

Y.Mukaigawa, S.Tagawa, J.Kim, R.Raskar, Y.Matsushita, and Y.Yagi(a) scene(b) synthetic
aperture(c) FHFI w/ofactorization(d) textureterm(e) groundtruth(f) views under normal illumination(g) focused high frequency illumination(h) masking term(i) attenuation term

Fig. 10. Result of the combination of the FHFI and the factorization.

orange mesh covers a textured paper and (f) shows all views from the virtual
cameras under normal illumination1 . By simply averaging these views, a synthetic aperture image can be generated as shown in (b). Although the obstacleis blurred, the orange color of the mesh affects the paper.
The mesh becomes dark by the FHFI because it is not illuminated, while
the paper is bright as shown in (g). By averaging these views, the dark meshis blurred and the orange color correctly disappears as shown in (c). However,there are uneven dark regions due to attenuation. The factorization decomposes
the observed views (g) into the masking term (h), the attenuation term (i), andthe texture term (d). We can see that the attenuation can be reduced especiallyaround the letter of the black ‘A’ and the red ‘D’, since the occlusion due to themesh is regarded as masks.
1

Although there are fifty mirror patches, only forty eight patches were used because
two patches were misaligned.

Hemispherical Confocal Imaging using Turtleback Reflector

813Limitations– The resolution of the virtual projectors and cameras is low because theimaging areas of real projectors and cameras are divided for virtual ones.
– The observable area is narrow because all projectors must illuminate and allcameras must observe a common area. To enlarge the area, a large turtle
reflector is necessary and it may be difficult to construct.
– The factorization is basically an ill-posed problem. For example, we can notdistinguish two different scenes in which red texture is covered with colorless
sheet and white texture is covered with red sheet. Some empirical constraints
such as the smoothness of the attenuation are necessary.

9Conclusion

We propose a new method of the hemispherical confocal imaging. This newimaging technique enables us to observe clear views of a particular depth in a
3-D scene. The originally designed turtleback reflector can divide the imaging
areas so that a projector and a camera mimic a number of virtual projectors andcameras surrounding the scene. The combination of the focused high frequency
illumination and the factorization can illuminate only the particular depth andeliminate scattering and attenuation. We have constructed a prototype systemand confirmed the principles of the hemispherical aperture, descattering, andfactorization.
One of our future works is to rebuild a more accurate optical device using ahigh resolution projector and evaluate the total performance, because the current prototype system only showed the principles separately. To develop some
applications which can visualize any cross-sectional views of a translucent object
is important. Another future work is to visualize the inside of the human bodyusing infrared light.
Acknowledgement
This work has been supported by a Microsoft Research CORE5 project andGrants-in-Aid for Scientific Researches (21680017 and 21650038).

References
1. Adelson, E. H., Wang, J. Y. A.: Single Lens Stereo with a Plenoptic Camera. IEEE
Tran. on PAMI (1992) 99–106
2. Cossairt, O., Nayar, S. K., Ramamoorthi, R.: Light Field Transfer: Global Illumination Between Real and Synthetic Objects. Proc. SIGGRAPH2008 (2008) 1–6
3. Dana, K. J., Wang, J.: Device for convenient measurement of spatially varying
bidirectional reflectance. J. Opt. Soc. Am. A, Vol.21, Issue 1 (2004) 1–12
4. Debevec, P., Hawkins, T., Tchou, C., Duiker, H. P., Sarokin, W., Sagar, M.: Acquiring the Reflectance Field of a Human Face. Proc. SIGGRAPH2000 (2000) 145–156

14

Y.Mukaigawa, S.Tagawa, J.Kim, R.Raskar, Y.Matsushita, and Y.Yagi

5. Fuchs, C., Heinz, M., Levoy, M., Seidel, H., Lensch, H.: Combining Confocal Imaging and Descattering. Proc. Computer Graphics Forum, Special Issue for the Eurographics Symposium on Rendering 27, 4 (2008) 1245–1253
6. Garg, G., Talvala, E. V., Levoy, M., Lensch, H. P. A.: Symmetric Photography :Exploiting Data-sparseness in Reflectance Fields. Proc. EGSR2006 (2006) 251–262
7. Ghosh, A., Achutha, S., Heidrich, W., O’Toole, M.: BRDF Acquisition with Basis
Illumination. Proc. ICCV2007 (2007)
8. Gu, J., Nayar, S. K., Grinspun, E., Belhumeur P. N., Ramamoorthi, R.: Compressive Structured Light for Recovering Inhomogeneous Participating Media. Proc.
ECCV2008 (2008) 845–858
9. Han, J. Y., Perlin, K.: Measuring Bidirectional Texture Reflectance with a Kaleidoscope. ACM Transactions on Graphics, Vol.22, No.3 (2003) 741–748
10. Kuthirummal, S., Nayar, S. K.: Multiview Radial Catadioptric Imaging for Scene
Capture. Proc. SIGGRAPH2006 (2006) 916–923
11. Levoy, M., Chen, B., Vaish, V., Horowitz, M., McDowall, I., Bolas, M.: Synthetic
Aperture Confocal Imaging. Proc. SIGGRAPH2004 (2004) 825–834
12. Levoy, M., Hanrahan, P.: Light field rendering. Proc. SIGGRAPH’96 (1996) 31–42
13. Masselus, V., Peers, P., Dutré, P., Willems, Y. D.: Relighting with 4D incident
light fields. Proc. SIGGRAPH2003 (2003) 613–620
14. Matusik, W., Pfister, H., Ngan, A., Beardsley, P., Ziegler, R., McMillan, L.: ImageBased 3D Photography using Opacity Hulls. Proc. SIGGRAPH2002 (2002) 427–437
15. Minsky, M.: Microscopy apparatus. US Patent 3013467 (1961)
16. Mukaigawa, Y., Sumino, K., Yagi, Y.: Multiplexed Illumination for Measuring
BRDF using an Ellipsoidal Mirror and a Projector. Proc. ACCV2007 (2007) 246–257
17. Müller, G., Bendels, G. H., Klein, R.: Rapid Synchronous Acquisition of Geometry
and Appearance of Cultural Heritage Artefacts. Proc. VAST2005 (2005) 13–20
18. Narasimhan, S. G., Nayar, S. K., Sun, B., Koppal, S. J.: Structured light in scattering media. Proc. ICCV2005, Vol 1 (2005) 420–427
19. Nayar, S. K., Krishnan, G., Grossberg, M. D., Raskar, R.: Fast Separation of
Direct and Global Components of a Scene using High Frequency Illumination. Proc.
SIGGRAPH2006 (2006) 935–944
20. Sen, P., Chen, B., Garg, G., Marschner, S., Horowitz, M., Levoy, M., Lensch, H.:
DualPhotography. Proc. SIGGRAPH2005 (2005) 745–755
21. Treibitz, T., Schechner, Y. Y.: Active Polarization Descattering. IEEE Tran. on
PAMI, Vol 31, Issue 3 (2009) 385–399
22. Unger, J., Wenger, A., Hawkins, T., Gardner, A., Debevec, P.: Capturing and
Rendering With Incident Light Fields. Proc. EGRW 2003 (2003) 141–149
23. Vaish, V., Szeliski, R., Zitnick, C. L., Kang, S. B., Levoy, M.: Reconstructing
Occluded Surfaces using Synthetic Apertures: Stereo, Focus and Robust Measures.
CVPR2006, Vol.II (2006) 2331–2338
24. Vaish, V., Wilburn, B., Joshi, N., Levoy, M.: Using Plane + Parallax for Calibrate
Dense Camera Arrays. Proc. CVPR 2004 (2004)
25. Veeraraghavan, A., Raskar, R., Agrawal, A., Mohan, A., Tumblin, J.: Dappled
Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded
Aperture Refocusing. Proc. SIGGRAPH2007 (2007)
26. Ward, G. J.: Measuring and Modeling anisotropic reflection. Proc. SIGGRAPH’92(1992) 255–272