2019 IEEE Winter Conference on Applications of Computer Vision

Multispectral Direct-Global Separation of Dynamic Scenes
Moriaki Torii
Takahiro Okabe
Kyushu Institute of Technology

Toshiyuki Amano
Wakayama University

okabe@ai.kyutech.ac.jp

amano@sys.wakayama-u.ac.jp

Abstract
In this paper, we propose a method for separating direct
and global components of a dynamic scene per illumination
color by using a projector-camera system; it exploits boththe color switch and the temporal dithering of a DLP projector. Our proposed method is easy-to-implement because
it does not require any self-built equipment and temporalsynchronization between a projector and a camera. In addition, our method automatically calibrates the projectorcamera correspondence in a dynamic scene on the basis of
the consistency in pixel intensities, and optimizes the projection pattern on the basis of noise propagation analysis.
We implemented the prototype setup and achieved multispectral direct-global separation of dynamic scenes in 60
Hz. Furthermore, we demonstrated that our method is effective for applications such as image-based material editing and multispectral relighting of dynamic scenes where
wavelength-dependent phenomena such as ﬂuorescence areobserved.

1. Introduction
When a scene is illuminated by a light source, the radiance observed at each point in the scene consists of two
components: direct and global [11]. The direct component
is caused by the direct illumination from the light source,
while the global component is caused by the illuminationfrom the other points in the scene due to interreﬂection,
subsurface scattering, volumetric scattering, and diffusion.
Separating direct and global components of scenes is important for various computer vision techniques such as shaperecovery, image-based material editing, and improving image quality [11, 6].
In this paper, we propose a method for separating direct
and global components of a dynamic scene per illumination
color by using a projector-camera system. The direct-global
separation per illumination color, e.g. per primary color ofa projector, is useful for studying the interactions highly
dependent on the wavelength of light: refraction, diffraction, scattering, and ﬂuorescence [2, 18]. Our proposed
978-1-7281-1975-5/19/$31.00 ©2019 IEEE
DOI 10.1109/WACV.2019.00209

method enables us to investigate dynamic scenes wheresuch wavelength-dependent phenomena are observed.
It is known that one can separate direct and global components in a scene by projecting high-frequency patterns
from a projector, because the global components are lowfrequency in general [11]. Therefore, the multispectral
direct-global separation of static scenes can be realized byprojecting several high-frequency patterns with different illumination colors. Unfortunately, however, such a straightforward method is not applicable to dynamic scenes, since
the frame rates of off-the-shelf projectors are low in general.
The key idea of our proposed method is to exploit boththe color switch [7] due to a color wheel and the temporal dithering [10] due to a DMD (Digital Mirror Device)of a DLP (Digital Light Processing) projector. We makeuse of such a projector for rapidly changing illumination
conditions in a dynamic scene of interest. Then, the image sequence of the scene is captured by using a high-speed
camera so that the effects of the rapidly-varying illumination conditions on the scene are observed.
Speciﬁcally, our proposed method studies the image sequence of a scene when the input to a DLP projector is asingle high-frequency pattern. Note that the output from the
projector rapidly changes in both colors and intensities dueto the color switch and the temporal dithering even for a single input pattern. Our method uses a reference object, andthen does not require temporal synchronization between a
projector and a camera. In addition, our method automatically calibrates the projector-camera correspondence in a
dynamic scene on the basis of the consistency in pixel intensities, and optimizes the projection pattern on the basisof noise propagation analysis. We make use of the 3D modein the side-by-side format of a projector, and then achieve
multispectral direct-global separation in 60 Hz.
We implemented the prototype setup and conﬁrmed theeffectiveness of our proposed method. In particular, we
showed that our proposed auto-calibration of projectorcamera correspondence and optimization of projection pattern work well. We demonstrated that our method is effective for applications such as image-based material editing and multispectral relighting of dynamic scenes where
1923wavelength-dependent phenomena such as ﬂuorescence areobserved.
The main contribution of this study is threefold. First,
we achieve fast multispectral direct-global separation by
exploiting both the color switch and the temporal dithering of a DLP projector. Second, our proposed method iseasy-to-implement, because it uses a consumer DLP projector and a consumer high-speed camera, and does not
require any self-built equipment. In addition, our method
achieves auto-calibration of projector-camera correspondence and does not require projector-camera temporal synchronization. Third, our method is robust, because the highfrequency projection pattern is optimized on the basis ofnoise propagation analysis.
The rest of this paper is organized as follows. In Section 2, we brieﬂy summarize related work. In Section 3,
a method for multispectral direct-global separation of dynamic scenes is proposed. We report the experimental results and the applications in Section 4 and Section 5. We
present concluding remarks in Section 6.

2. Related Work2.1. Direct-global separation
The relationship between the light sources illuminatinga scene and its images taken under those light sources iscalled light transport (LT). The LT reveals where the lightradiated from a certain light source interacts with the scene
just before arriving at a camera. Acquiring the LT of a scene
is important for various applications such as image-based
relighting [4, 12, 16, 17] and controlling appearance of thescene [14, 5, 3, 1].
On the other hand, the direct-global separation reveals
how the light radiated from a light source interacts with ascene. Nayar et al. [11] propose a method for separating
direct and global components in a scene by projecting highfrequency patterns from a projector on the basis of the insight that global components are low-frequency in general.
Separating direct and global components of scenes is effective for various computer vision techniques such as shaperecovery, image-based material editing, and improving image quality.
Gu et al. [6] extend the above direct-global separation
for a single light source to that for multiple light sources.
Speciﬁcally, they optimize a set of high-frequency projection patterns in terms of SNR (signal-to-noise ratio) on thebasis of illumination multiplexing [15], and then show that
it is effective for scene recovery. Mukaigawa et al. [9] showthat projecting high-frequency patterns is useful also forstudying the light transport in scattering media, in particular for decomposing multiple scattering into each bouncecomponent.
In contrast to the above existing methods, we propose

a method for separating direct and global components of a
dynamic scene per illumination color by using a projectorcamera system. As demonstrated in Section 5, multispectral
direct-global separation enables applications such as imagebased material editing and multispectral relighting of dynamic scenes where wavelength-dependent phenomena areobserved.

2.2. Active illumination using DLP projector
In a single-chip DLP projector, a color wheel consisting of three (or more) color ﬁlters with different spectral
transmittances is placed between a white light source anda DMD. The color wheel rotates at high speed in front of
the light source, and generates various colors by mixing thetransmitted lights with different SPDs (Spectral Power Distributions) via time-division multiplexing. Then, the DMD
controls the intensity of the light transmitted from the color
wheel by temporal dithering; it switches micro mirrors “on”
and “off” directions at high speed.
Narasimhan et al. [10] exploit the rapidly-varying illumination intensities due to the temporal dithering, and thenpropose a method for separating direct and global components of dynamic scenes by using a high-speed camera.
They show that the temporal dithering is useful also forfast active vision such as structured light-based range ﬁnding and photometric stereo. Unfortunately, however, their
method assumes that the illumination color is ﬁxed, andthen it cannot separate direct and global components per
illumination color.
On the other hand, Maeda and Okabe [8] exploit the
rapidly-varying illumination colors due to the color switchof a multi-primary DLP projector, and then propose amethod for separating direct and global components of astatic scene per primary color of the projector by using ausual camera with a short exposure time. In addition, they
propose a method for estimating the SPDs of the primarycolors in a non-destructive manner. Unfortunately, however,their method assumes static scenes, and then it is not applicable to objects in motion. Han et al. [7] propose a method
for recovering the spectral reﬂectance of a moving object byusing a DLP projector and a high-speed camera on the basisof the color switch.
Our proposed method exploits both the color switch andthe temporal dithering of a DLP projector, and achieves the
direct-global separation of dynamic scenes per illumination
color. Our method is not a straightforward combination of
the above existing methods based on the temporal dithering [10] and the color switch [8]. In particular, we realize
auto-calibration of projector-camera correspondence on thebasis of the consistency in pixel intensities. Furthermore,
we optimize the high-frequency projection pattern on thebasis of noise propagation analysis so that it is robust to
noises.

1924

3. Proposed Method
3.1. Principle of direct-global separation
Nayar et al. [11] propose a method for direct-global separation of a static scene by using a projector-camera system; it makes use of the properties of global components,
i.e. they are low-frequency in general, and studies the images of the scene under high-frequency projection patterns.
Speciﬁcally, their simplest method uses two images; they
are captured when a black-and-white checkered pattern or
its negative-positive reversed pattern are projected from a
projector to the scene.
Let us denote the two output intensities from a projector corresponding to white and black input intensities of acheckered pattern by a and b, e.g. a = 1 and b = 01 . The
pixel intensities observed at a certain point in the scene are
given by
I+

=

I−

=

1
aLd + (a + b)Lg ,2
1
bLd + (a + b)Lg ,
2

(b)

(c)

(d)

(e)

(a)

Figure 1. The color switch and the temporal dithering: (a) the input
gray-scale image and (b)(c)(d)(e) the example images of the white
reﬂectance standard illuminated by a DLP projector and captured
by a high-speed camera, when the gray-scale image is input to theprojector.

(1)
(2)

where Ld and Lg are the direct and global components at the
point, and I + and I − are the larger and smaller pixel intensities observed there when a > b [11, 10]. Therefore, one
can compute the direct component Ld and the global component Lg by solving the simultaneous linear equations ineq.(1) and eq.(2), when the pixel intensities I + and I − andthe output intensities from the projector a and b are known.

3.2. Multispectral separation of dynamic scene
According to the principle of the direct-global separation described in the previous subsection, the multispectral direct-global separation of static scenes can be realized
by projecting at least 6 (= 3 primary colors × 2 negativepositive patterns) high-frequency patterns with different illumination colors2 . Unfortunately, however, the frame rates
of off-the-shelf projectors are low in general, and then theabove straightforward method causes motion blur for dynamic scenes. In addition, the temporal synchronization
between a projector and a camera is required.
Our proposed method captures the image sequence of a
dynamic scene illuminated by a consumer DLP projector byusing a consumer high-speed camera, and then studies theeffects of the rapidly-varying illumination conditions due
to both the color switch and the temporal dithering on thescene. Figure 1 (a) shows the input gray-scale image to a
DLP projector which consists of the blocks with different
pixel intensities from 0 (the upper left) to 255 (the lower
1 Note that the relationship between the input and output intensities ofa projector is not linear in general.
2 It results in the multispectral direct-global separation in 5 Hz when weuse a projector with 30 fps.

right). Figure 1 (b)(c)(d)(e) shows the example images of
the white reﬂectance standard illuminated by the projector
and captured by a high-speed camera, when (a) the grayscale image is input to the projector. We can see the effectsof the color switch in (b)(c)(d) and those of the temporaldithering in (d)(e).
In our proposed method, the input to a DLP projector isa single high-frequency pattern: a “checkered” pattern withtwo different gray-scale intensities. Our method captures
not only a dynamic scene of interest but also a reference
object, e.g. the white reﬂectance standard at the same time
as shown in Figure 4. This setup enables us to classify thecaptured images into the images illuminated only by eachprimary color of the projector on the basis of the observedcolors on the reference object. We assume that the scene
is stationary within a single frame of the result image sequence, e.g. 1/60 sec in our current implementation. Therefore, our multispectral direct-global separation results in the
direct-global separation of a stationary scene illuminated bya single primary color of the projector.
Then, let us consider the N images of a stationary scene
illuminated by a single illumination color. In a similarmanner to eq.(1) and eq.(2), the pixel intensity In (n =1, 2, 3, ..., N ) observed at a certain point in the scene isgiven by1
In = an Ld + (an + bn )Lg
(3)
2
or
1
In = bn Ld + (an + bn )Lg(4)
2
in the n-th image. Here, Ld and Lg are the direct and global
components at the point, and an and bn are the two output
intensities from the projector. Although the output inten-

1925

half mirror

cameramovingsurfacecameramovingsurface(a)projector

projector

(a)

(b)

Figure 2. The projector-camera correspondence in a dynamic
scene: (a) dynamically varying correspondence and (b) ﬁxed correspondence using a half mirror.

sities an and bn vary due to the temporal dithering, we canestimate those intensities also from the observed pixel intensities on the reference object. Therefore, one can computethe direct component Ld and the global component Lg bysolving the simultaneous linear equations in eq.(3) or thosein eq.(4), if whether the point is directly illuminated by anor bn is known.

3.3. Auto-calibration of pro-cam correspondence
The multispectral direct-global separation described in
the previous subsection suffers from an ambiguity, when the
projector-camera correspondence is unknown. As shown in
Figure 2 (a), in dynamic scenes, the projector-camera correspondence also varies dynamically. Therefore, we cannot
distinguish whether we should solve the simultaneous linear equations in eq.(3) or those in eq.(4). To cope with the
problem of such an ambiguity, Narasimhan et al. [10] ﬁx
the projector-camera correspondence by using a half mirror
as shown in Figure 2 (b) and calibrate the correspondence
in advance.
In contrast to their method, our proposed method resolves the problem of the ambiguity on the basis of the consistency in pixel intensities. Speciﬁcally, we solve the both
simultaneous linear equations via the linear least-square
method asN 

1In − an Ld − (an + bn )Lg
2{Ld ,Lg }
n=1
min

or

N 


1
In − bn Ld − (an + bn )Lgmin2{Ld ,Lg }
n=1

(b)

(c)

Figure 3. The 3D mode in the side-by-side format: (a) the input
image to and (b)(c) the output images from a projector.

2(5)

Thus, compared with Narasimhan et al. [10], our proposed method with the auto-calibration of the projectorcamera correspondence does not require any additional
equipment such as a half mirror and does not require cumbersome pre-calibration of the projector-camera correspondence. In addition, our method enables us to move a projector and a camera independently during shooting.

3.4. Optimization of projection pattern
The output intensities from a DLP projector, i.e. how
the output intensities vary due to the color switch and thetemporal dithering depends on the input intensities to theprojector. Therefore, we can optimize the two input intensities of the checkered pattern in order to separate direct andglobal components accurately. We take the noises in theobserved pixel intensities into consideration, and then optimize those input intensities on the basis of the condition
numbers [13].
Speciﬁcally, we choose two blocks from the gray-scale
image in Figure 1 (a), and denote the pixel intensities
observed at the corresponding points on the white reﬂectance standard in the n-th image by an and bn (n =1, 2, 3, ..., N ). Then, we compute the condition numbers of
the following matrices:⎛⎞(a1 + b1 )/2
a1⎜ a2(a2 + b2 )/2 ⎟
⎜
⎟
(7)
⎜ ..
⎟
..
⎝ .
⎠
.
aN (aN + bN )/2
and

2
(6)

with the non-negativity constraints: Ld ≥ 0 and Lg ≥ 0.
Then, we choose the solution with the smaller sum of thesquared errors. In other words, our method is based on
the insight that the sum of the squared errors is smaller (orlarger) for the correct (or wrong) projector-camera correspondence.

⎛

b1
⎜ b2
⎜
⎜ ..
⎝ .

(a1 + b1 )/2(a2 + b2 )/2
..
.

bN

(aN + bN )/2

⎞
⎟
⎟
⎟.
⎠

(8)

Those condition numbers tell how much noises in the observations In propagate to the unknowns to be estimated Ld
and Lg . We compute the condition numbers for all the pairsof the input intensities in 8 bit (from 0 to 255), and thenselect the optimal pair of the input intensities for which thesum of the condition numbers with respect to RGB colors
is minimal. Note that smaller condition number is better.

1926

high-speed camera
(e)
(a)
(c)

(b)

(d)
DLP projector

Figure 4. Our setup: the ﬁrst scene is illuminated by a DLP projector and captured by a high-speed camera: (a) a wood ball, (b)a plastic cup, (c) slime, (d) wax in a bottle, and (e) the white reﬂectance standard (reference object).

(a)the projector outputs (b) and (c) in turn3 . Since we can distinguish the two spatially-shifted checkered patterns fromthe images of a reference object, we can realize the directglobal separation by using spatially-shifted projection patterns in a similar manner to the previous subsections.

4. Experiments
We implemented the prototype setup and achieved multispectral direct-global separation of dynamic scenes in 60
Hz. Please see our supplementary material for the result
image sequences.

4.1. Experimental setup
In our experiments, we used a DLP projector NP-VE282
from NEC and a high-speed camera Fastcam Mini UX50
from Photron. The frame rate and the exposure time were
set to 10,000 fps and 0.05 ms respectively. We conducted
multispectral direct-global separation by using our proposed method for every 166 images, i.e. in 60 Hz.
The captured images were classiﬁed into the images illuminated only by each primary color, i.e. R, G, and B in ourcurrent setup by using the mean shift clustering. For solving
eq.(5) or eq.(6), we used the images classiﬁed into each primary color. As described in Subsection 3.4, we optimized
the two pixel intensities of the input checkered pattern, andthen set them to 134 and 172.

4.2. Experimental results(b)
Figure 5. The captured images of the ﬁrst scene: (a) the image ofthe scene illuminated by uniform projection pattern and (b) those
illuminated by checkered projection patterns.

3.5. Use of 3D mode in side-by-side format
The simplest direct-global separation described in Subsection 3.1 uses two projection patterns; one is a black-andwhite checkered patten and the other is its negative-positive
reversed pattern. It is known that the simplest method often causes visible artifacts near the boundaries between theblocks of the checkered pattens because the spatial resolutions of a projector and a camera are limited. Nayar etal. [11] show that those artifacts can be reduced by using
more than two projection patterns, e.g. spatially shifted patterns from the original checkered patten.
Accordingly, our proposed method makes use of the
3D mode in the side-by-side format, which is relatively
common function even for consumer projectors, and thenprojects two spatially-shifted checkered patterns in practice.
When a single image in Figure 3 (a) is input to the projector,

Figure 4 shows the ﬁrst scene which includes (a) a woodball, (b) a plastic cup, (c) slime, (d) wax in a bottle, and (e)the white reﬂectance standard (reference object). The slime
was running down into the cup during shooting. Figure 5
shows (a) the image of the scene illuminated by uniform
projection pattern and (b) those illuminated by checkered
projection patterns. We can see that the spatially-shifted
checkered patterns with varying illumination colors and intensities are projected to the scene.
Figure 6 shows the result of our proposed method for theﬁrst scene: (a) the direct and (b) the global components of acertain frame of the result image sequence and (c)(d) thoseof another frame. The illumination colors are R, G, and B
from top to bottom. We can see that the direct components
include the specular reﬂection components on the cup andthe bottle and the diffuse reﬂection components on the balland the white reﬂectance standard. On the other hand, we
can see that the global components include the subsurface
scattering on the wax and the slime. Interestingly, we canobserve that the slime radiates yellow light under blue illumination; it contains ﬂuorescent material which absorbs
3 In our current setup, the left-eye and the right-eye images quickly
switch in 120 Hz. For the sake of simplicity, we do not take account of
the nonlinear mapping from the input intensity to the output intensity inthis illustration.

1927

(a)

(b)

(c)

(d)

Figure 6. The result for the ﬁrst scene: (a) the direct and (b) the
global components of a certain frame of the result image sequenceand (c)(d) those of another frame.

(a)

(b)

Figure 7. The result for the ﬁrst scene with ﬁxed correspondence:(a) the direct and (b) the global components of the same frame as
Figure 6 (c)(d).

incident light at a certain wavelength, and then emits light
at longer wavelengths than the incident one [2, 18]. Oneof the advantages of the multispectral direct-global separation is that it can capture wavelength-dependent phenomena
such as ﬂuorescent and translucent material.
Unfortunately, however, some artifacts are still visible mainly near the boundaries between the “white” and“black” blocks of the checkered projection pattern. The
main reason would be the limited spatial resolution of the

(a)

(b)

(c)

(d)

Figure 8. The result for the second scene: (a) the direct and (b) the
global components of a certain frame of the result image sequenceand (c)(d) those of another frame.

high-speed camera: 640 × 240. Another possible reason isthat the projector has a shallow depth of ﬁeld. For the above
reasons, those boundaries are blurred, and then the pixel intensities deviates from eq.(3) or eq.(4) near the boundaries.
We show the result of our proposed method for the second scene in Figure 8. The target object is a cloth (ﬂuorescent satin). Similar to the result for the ﬁrst scene, we canobserve ﬂuorescence; the cloth appears reddish under B and
G illumination colors. In addition, the interreﬂection among
wrinkled surfaces is observed in the global components.
Projector-camera correspondence: Figure 7 shows theresult when we assume a static scene. Speciﬁcally, we did
not conduct our auto-calibration of projector-camera correspondence, but ﬁxed the correspondence to that of the initial frame. When comparing the same frame of the result
image sequence, i.e. comparing Figure 6 (c)(d) with Figure 7 (a)(b), we can see that some artifacts are visible near
the upper half of the plastic cup in the result assuming astatic scene. Since the slime is there in the initial frame,
those artifacts are considered to be caused by object motion. Thus, we can conﬁrm the effectiveness of our proposed method, which conducts dynamical auto-calibrationof projector-camera correspondence.
Projection pattern: Figure 9 shows the results of thedirect-global separation, when we change the two pixel in-1928

Condition Number

RMS error

25

1000
R
G
B

20

R
G
B

15
100
10
5
10
1

10

100

1000

10000

1

10

100

Order
(a)

(a)

1000

10000

Order
(b)

Figure 10. The numerical analysis: (a) the condition number and(b) the RMS error for each primary color vs. the order of the pair.

(b)

we tested the best, the 100-th best, and the 10000-th best
pairs of pixel intensities from 256 C2 = 32, 640 candidates.
For fair comparison, we use a static scene, and consider
the result when we use 4 black-and-white checkered projection patterns in a similar manner to the original directglobal separation method [11] as the ground truth. We cansee qualitatively that the best and the 100-th best pairs work
well, but the result using the 10000-th best pair is signiﬁcantly degraded.
Figure 10 shows the numerical analysis: (a) the condition number and (b) the RMS error for each primary color
vs. the order of the pair. We can see that the RMS error iscorrelated with the condition number. Thus, we can conﬁrm
the effectiveness of our proposed method, which optimizes
a projection pattern on the basis of noise propagation analysis in terms of condition number.

(c)

5. Applications
We demonstrate that our proposed method is effective forimage-based material editing and multispectral relighting ofdynamic scenes where wavelength-dependent phenomena
are observed. Please see our supplementary material for the
result image sequences.

(d)Figure 9. The direct (left) and the global (right) components: (a)the ground truth and the results using (b) the best, (c) the 100-th
best, and (d) the 10000-th best projection patterns.

tensities of the checkered projection pattern. Speciﬁcally,

Image-based material editing: Figure 11 (a) shows theoriginal images of a dynamic scene under white illumination. We can change the material perception of the scene byediting (b)(c) the ratio of the direct and global componentsand (d)(e) the color of ﬂuorescence.
Multispectral relighting: Figure 12 shows the result of
the multispectral relighting for (a) the direct and (b) the
global components. Our proposed method enables us tochange illumination color for each component by editing
the weights of R, G, B images. The multispectral relighting
for the direct and global components can be combined withimage-based material editing.

1929such as image-based material editing and multispectral relighting of dynamic scenes where wavelength-dependent
phenomena are observed. Our future study includes designing the color ﬁlters of the color wheel in a DLP projector,
i.e. optimizing the number and SPDs of the primary colors
for speciﬁc computer vision applications.

(a)

(b)

Acknowledgments(c)This work was partially supported by JSPS KAKENHI Grant Numbers JP18H05011, JP17H01766, andJP16H01676, and Support Center for Advanced Telecommunications Technology Research (SCAT).

(d)References(e)Figure 11. Application to image-based material editing: (a) theoriginal images under white illumination, (b)(c) the ratio of thedirect and global components and (d)(e) the color of ﬂuorescenceare edited.

(a)

(b)

Figure 12. Application to multispectral relighting: (a) the directand (b) the global components.

6. Conclusion and Future Work
We proposed a fast, robust, and easy-to-implement
method for separating direct and global components of a
dynamic scene per illumination color. Speciﬁcally, our proposed method exploits the rapidly-varying illumination conditions due to both the color switch and the temporal dithering of a DLP projector. We conducted a number of experiments and showed that our method, in particular our autocalibration of projector-camera correspondence and optimization of projection pattern work well. Furthermore, we
demonstrated that our method is effective for applications[1] T. Amano. Shading illusion: a novel way for 3-D representation on the paper media. In Proc. PROCAMS2012, 2012.
[2] K. Barnard. Color constancy with ﬂuorescent surfaces. In
Proc. CIC1999, pages 257–261, 1999.
[3] O. Bimber, D. Iwai, G. Wetzstein, and A. Grundhöfer. The
visual computing of projector-camera systems. Computer
Graphics Forum, 27(8):2219–2245, 2008.
[4] P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin,
and M. Sagar. Acquiring the reﬂectance ﬁeld of a humanface. In Proc. ACM SIGGRAPH 2000, pages 145–156, 2000.
[5] M. Grossberg, H. Peri, S. Nayar, and P. Belhumeur. Making
one object look like another: controlling appearance using a
projector-camera system. In Proc. IEEE CVPR2004, pages
I–452–459, 2004.
[6] J. Gu, T. Kobayashi, M. Gupta, and S. Nayar. Multiplexed
illumination for scene recovery in the presence of global illumination. In Proc. IEEE ICCV2011, pages 691–698, 2011.
[7] S. Han, I. Sato, T. Okabe, and Y. Sato. Fast spectral reﬂectance recovery using DLP projector. IJCV, 110(2):172–
184, 2014.
[8] K. Maeda and T. Okabe. Acquiring multispectral light transport using multi-primary DLP projector. In Proc. IPTA2016,
pages 1–6, 2016.
[9] Y. Mukaigawa, Y. Yagi, and R. Raskar. Analysis of lighttransport in scattering media. In Proc. IEEE CVPR2010,
pages 153–160, 2010.
[10] S. Narasimhan, S. Koppal, and S. Yamazaki. Temporaldithering of illumination for fast active vision. In Proc.
ECCV2008, pages 830–844, 2008.
[11] S. Nayar, G. Krishnan, M. Grossberg, and R. Raskar. Fast
separation of direct and global components of a scene using high frequency illumination. In Proc. ACM SIGGRAPH
2006, pages 935–944, 2006.
[12] P. Peers, D. Mahajan, B. Lamond, A. Ghosh, W. Matusik,
R. Ramamoorthi, and P. Debevec. Compressive light transport sensing. ACM TOG, 28(1):Article No.3, 2009.
[13] W. Press, S. Teukolsky, W. Vetterling, and B. Flannery. Numerical Recipes in C 3rd Edition: The Art of Scientiﬁc Computing. Cambridge University Press, 2007.
[14] R. Raskar, G. Welch, K.-L. Low, and D. Bandyopadhyay.
Shader lamps: animating real objects with image-based illumination. In Proc. EWRT2001, pages 89–102, 2001.

1930

[15] Y. Schechner, S. Nayar, and P. Belhumeur. A theory of multiplexed illumination. In Proc. IEEE ICCV2003, pages 808–
815, 2003.
[16] P. Sen and S. Darabi. Compressive dual photography. Computer Graphics Forum, 28(2):609–618, 2009.
[17] J. Wang, Y. Dong, X. Tong, Z. Lin, and B. Guo. Kernel
Nyström method for light transport. In Proc. ACM SIGGRAPH 2009, page Article No.29, 2009.
[18] C. Zhang and I. Sato. Image-based separation of reﬂective
and ﬂuorescent components using illumination variant andinvariant color. IEEE Trans. PAMI, 35(12):2866–2877, 2013.

1931