Separating Fluorescent and Reﬂective Components by Using a Single
Hyperspectral Image
Yinqiang Zheng
National Institute of Informatics

Ying Fu
The University of Tokyo

Antony Lam
Saitama University

yqzheng@nii.ac.jp

fuying@iis.u-tokyo.ac.jp

antonylam@cv.ics.saitama-u.ac.jp

Imari Sato
National Institute of Informatics

Yoichi SatoThe University of Tokyo

imarik@nii.ac.jp

ysato@iis.u-tokyo.ac.jp

Abstract
This paper introduces a novel method to separate fluorescent and reflective components in the spectral domain.
In contrast to existing methods, which require to capture
two or more images under varying illuminations, we aim
to achieve this separation task by using a single hyperspectral image. After identifying the critical hurdle in singleimage component separation, we mathematically design the
optimal illumination spectrum, which is shown to contain
substantial high-frequency components in the frequency domain. This observation, in turn, leads us to recognize a key
difference between reflectance and fluorescence in responseto the frequency modulation effect of illumination, which
fundamentally explains the feasibility of our method. On
the practical side, we successfully find an off-the-shelf lamp
as the light source, which is strong in irradiance intensity
and cheap in cost. A fast linear separation algorithm isdeveloped as well. Experiments using both synthetic data
and real images have confirmed the validity of the selected
illuminant and the accuracy of our separation algorithm.

1. Introduction
Hyperspectral information has proven to be usefulin such applications as high-quality color reproduction,
spectra-based material identification, computer-aided medical diagnostics and so on. A great variety of hyperspectral
imaging methods and apparatuses (see [3, 11, 14, 19] andmany others) have been developed and implemented to capture spectral reflectance of scenes.
Fluorescence is a common optical phenomenon present
in many natural objects, like coral, living organic tissues
and minerals. Fluorescent substances have also been widely
applied to paper and cloth for color augmentation, to cur-(a) Input(b) Reflectance(d) Household LED(c) Fluorescence(e) UV

Figure 1. Given a single hyperspectral image of some rocks (a)under the selected illuminant, our proposed algorithm can separate the fluorescent component (c) from the reflective one (b). As
reference, (d) shows the same scene under a household LED lamp,
while (e) shows the scene under a UV light. Since the fluorescentcomponent in these rocks can only be excited under UV irradiance,(d) almost contains the reflective component only.

rency notes for anti-counterfeiting, as well as to biochemical dyes for molecular labeling and tracking.
When the scene contains fluorescence, existing hyperspectral reflectance estimation methods, like those in [11,19], can not provide accurate estimation. This is because of the well-known wavelength shifting effect of fluorescence that, unlike a reflective surface, fluorescent substances would absorb incident irradiance and re-emit it atlonger wavelengths. Considering that fluorescent and reflective components interact with the illumination in quite
different ways, it is necessary to separate them from image
observations [4, 8, 9, 13, 15].
Existing works [6, 8, 15, 22, 23, 26] have addressed thisspectra separation problem, all of which require to capture13523

two or more images of the same scene under varying illuminations. Given a single image, is it possible to separate
the fluorescent and reflective components? To explore thisproblem is not only conceptually interesting but also practically rewarding. For example, it might simplify the image
capturing process and eliminate the burden of switching between illuminations.
In this work, we show, for the first time to our knowledge, the separation is feasible when provided a hyperspectral image captured under an illumination with highfrequency components, see Fig.1 for an example of fluorescent rocks. We start our analysis from a purely mathematical viewpoint, by first introducing the standard lowdimensional linear basis expression for the reflectance andfluorescent emission spectra to reduce the number of variables, and then retrieving the optimal illumination spectrum so as to conquer the dependency issue of these twobases. We examine the Fourier magnitude distribution of
the retrieved optimal illumination, and find that it contains
substantial high-frequency components. This observation
inspires us to investigate the underlying reason why highfrequency components are needed for separation, and finally leads us to recognize a key difference between reflectance and fluorescence in response to the frequency
modulation effect of illumination. Specifically, the Fourier
magnitude distribution of the reflective component will
be modulated by the illumination spectrum, while that of
the fluorescent component is independent of illumination
Fourier magnitude distribution. This property of fluorescence fundamentally explains the feasibility of separationby using a single hyperspectral image. On the practicalside, we successfully find an off-the-shelf high-intensity
discharge (HID) lamp as the light source, which is strong
in irradiance intensity and cheap in cost. A fast linear separation algorithm is developed to handle high resolution images as well. Experiments using both synthetic data andreal images have confirmed the validity of the selected illuminant and the accuracy of our separation algorithm.
Our major contributions can be summarized as follows:(i). We numerically design the optimal illumination spectrum to conquer the dependency issue of the fluorescentemission and reflectance bases; (ii). We analytically disclose when and why fluorescence and reflectance separationby using a single hyperspectral image is feasible; (iii). We
find an off-the-shelf lamp as the light source and develop an
efficient linear separation algorithm.
The remaining parts of this paper are organized as follows. In Sec.2, we give a more detailed literature review.
Sec.3 includes the numerical design of illuminantion spectrum, followed by the theoretical analysis and the linear separation algorithm. We show some experimental results in
Sec.4, and conclude this work in Sec.5.

2. Related Works
Due to its special wavelength shifting effect and color
constancy, fluorescence has recently received much attention in computer vision, along with various successful applications to shape recovery [12, 21, 24], camera photometric calibration [10] and inter-reflection removal [7].
Fluorescent objects contain both fluorescent and reflective components. Due to the difference in their mechanisms
when interacting with the illumination, it is necessary toseparate these two components from image observations.
This spectra separation task has shown to be critical forcolor relighting [8, 13, 15], coral reef monitoring [9] andcancer diagnosis in biomedical engineering [4]. In the following, we review the most closely related works on fluorescence and reflectance separation.
Thanks to the color invariance property of fluorescence,researchers in [25] have been able to separate fluorescencefrom reflectance by using two RGB images under different yet unknown illuminations via independent component
analysis. Rather than being satisfied with RGB colors offluorescence and reflectance, Fu et al. [6] have tried to recover their spectra by using as many as nine RGB images
captured under varying color illuminations.
In addition to wideband RGB cameras, hyperspectral
imaging devices have also been widely used for fluorescence and reflectance spectra separation. For example, the well-known bispectral method [16] coordinates a
monochromator and a spectrometer to measure the reflective and fluorescent spectra of a single scene point in an
exhaustive manner. Lam and Sato [15] used a hyperspectral image sensor instead, and introduced the linear basis
expression of reflectance and fluorescence spectra so as
to reduce the required number of measurements, although
about twenty images are still needed. The bispectral coding scheme [22] is rooted in the classical bispectral measurement method as well, in which dozens of hyperspectral
images have to be captured under shifting narrow-band illuminations.
To further reduce the number of images, Tominaga et al.
[23] implicitly assumed that the scene contains no blue fluorescence, and successfully separated these two components
by using two hyperspectral images under ordinary illuminants. Fu et al. [8] relaxed this assumption, at least in principle, by using two high-frequency and complementary sinusoid illuminations defined in the spectral domain. The reason of using such specialized illuminations lies in their demanding requirement that the fluorescent absorption scalars
under two different illuminations should be the same. The
resulting separation algorithm is very simple and its accuracy is outstanding. Unfortunately, the specialized illumination spectra were generated by a programmable light sourcein [8], which is expensive in cost and weak in irradiance intensity. As a result, one has to reduce the camera frame rate3524

so as to capture a bright enough image. Zheng et al. [26]introduced one more hyperspectral image, and were able touse ordinary illuminants instead, but at the added cost ofsolving a more complicated bilinear problem via iterations.
To use multiple images under varying illuminations complicates the data capturing process (e.g. synchronization operation of the camera and illumination). This motivates us
to explore the possibility of using a single hyperspectral image for this separation task. In the following, we are going
to examine requirements of the illumination spectrum, look
for an off-the-shelf lamp as the illuminant and develop a fast
linear algorithm for separation.

3. Fluorescence and Reﬂectance Separation ofa Hyperspectral Image

to a scale ambiguity between s and e, we can without lossof generality merge them into ê. Our objective is to separatea hyperspectral measurement i into the reflectance part r
and the fluorescent part ê, by using a proper illumination
spectrum l.
The number of constraints in Eqn.2 is n, thus we can notdirectly determine 2n variables in r and ê. A straightforward idea to resolve this issue is to introduce some lowdimensional models so as to reduce the number of variables. Both linear and nonlinear low-dimensional models
have been suggested for reflectance [17, 20] and fluorescentemission spectra [15, 26]. Due to its simplicity, we choose
the linear low-dimensional expression. By embedding linear basis models into Eqn.2, we obtain[
α + Bββ = LAi = Lr + ê = LAα3.1. Hyperspectral Fluorescence Imaging Model
For a pure reflective surface, the radiance i1 (λ) at wavelength λ is a direct product of the irradiance l(λ) and thereflectance coefficient r(λ), i.e., i1 (λ) = l(λ)r(λ).
In contrast, a pure fluorescent surface would first absorb
some energy in shorter wavelengths and re-emit it in longer
wavelengths. According to [8, 25], this(∫shifting effect) canbe approximately described by i2 (λ) =l(λ̃)a(λ̃)dλ̃ e(λ),
in which a(λ) and e(λ) denote the absorption and emission
intensity at λ, respectively.
Let us recall that a usual fluorescent object contains both
the reflective and fluorescent components. Therefore, thehyperspectral imaging equation of a fluorescent-reflective
surface reads(∫)i(λ) = i1 (λ) + i2 (λ) = l(λ)r(λ) +l(λ̃)a(λ̃)dλ̃ e(λ). (1)Given a proper illumination spectrum l=]Tlλ1 , lλ2 , · · · , lλnand its corresponding radiance of a[]Tscene point i = iλ1 , iλ2 , · · · , iλn recorded by a hypersepctral device with n bands, the imaging equation in Eqn.1 canbe rewritten as(∫)
i = Lr +l(λ̃)a(λ̃)dλ̃ e = Lr + se = Lr + ê, (2)[[]Twhere L = diag(lλ1 , lλ2 , · · · , lλn ), and r = rλ1 , rλ2 , · · · , rλn]T[denotes the reflectance spectrum. e = eλ1 , eλ2 , · · · , eλnrepresents the fluorescent emission spectrum. We have assumed that the response function of the hyperspectral device
has been normalized.
Note that the absorption spectrum a=[]Taλ1 , aλ2 , · · · , aλnis also important in describing thespectral behavior of fluorescence. From Eqn.2, we seethat ∫it simply integrates with the light source to a scalars = l(λ̃)a(λ̃)dλ̃ that scales the emission spectrum e. Due

][
B αT

βT

]T

,

(3)

where An×u and α u×1 are the reflectance bases and coefficients, while Bn×v and β v×1 the bases and coefficients forfluorescent emission. u and v denote the number of bases forreflectance and fluorescent emission, respectively. We canuse the principle component analysis (PCA) of the training
data to determine data-dependent bases A and B1 . According to [15, 17, 20], the number of reflectance bases u is usually 8, while that of fluorescent emission bases v is about
12.
At first glance, Eqn.3 seems to be well-posed, since thenumber of constraints n is usually much greater than the
total number of bases (8 + 12 = 20). Unfortunately, as
recognized in [26], the bases A and B from PCA training
are not completely independent. In other words,
when
the
[
]
illumination is spectrally white, the matrix LA B degen[]erates to A B , whose condition number approaches in[]Tfinity. Under this condition, the coefficients α T β T can
not be uniquely determined, and the separation task is therefore infeasible. In the following, to resolve the challenging
basis dependency issue, we first mathematically design theillumination spectra by minimizing a condition number criterion.

3.2. Numerically Optimal Illumination Spectrum
Here, we borrow the concept in linear algebra that a linear equation becomes more robust as the condition number of the coefficient matrix reduces. Given the bases A∗
and B, we try to find an optimal illumination[] spectrum lsuch that the condition number of LA B is minimized,([])
that is, l∗ = argminl cond LA B . Hereafter, we denote[]
LA B as M(l) for brevity.
1 In all our experiments, we use PCA on the Munsell reflectance spectra
dataset [20] to obtain the reflectance bases A, while the McNamara fluorescent spectra dataset [18] to train the fluorescent emission bases B.

3525

1

1

1

1

1

1

0.8

0.8

0.8

0.8

0.8

0.8

0.6

0.6

0.6

0.6

0.6

0.6

0.4

0.4

0.4

0.4

0.4

0.4

0.2

0.2

0.2

0.2

0.2

0.2

0
0
0
0
0
0
420nm 490nm 560nm 630nm 700nm 420nm 490nm 560nm 630nm 700nm 420nm 490nm 560nm 630nm 700nm 420nm 490nm 560nm 630nm 700nm 420nm 490nm 560nm 630nm 700nm 420nm 490nm 560nm 630nm 700nm(a) OPTIMAL (1.8)(b) ELS (4.9)108(c) D2C (12.4)(d) OFFICE (55.2)(e) COLOR LEDs(74.0) (f) WHITE LED (301.5)

80

40

40

40

60

30

30

30

40

20

20

20

20

10

10

10

150

100

6
4

50

2
0
0

10

20

30

0
0

40

10

20

30

0
0

40

10

20

30

40

0
0

10

20

30

0
0

40

10

20

30

0
0

40

10

20

30

40

(g) Fourier Magnitude Distribution of Illumination Spectra from (a) to (f)
0.8

6
1
2
3
4
5
6
7
8

0.6
0.4

5
1
2
3
4
5
6
7
8

4

2
0.2
0
0

10

20

30

0
40 0

10

20

30

5
1
2
3
4
5
6
7
8

4
3

5
1
2
3
4
5
6
7
8

4
3

3

2

2

2

1

1

1

0
40 0

10

20

30

40

0
0

10

20

30

10
1
2
3
4
5
6
7
8

4

0
40 0

1
2
3
4
5
6
7
8

8
6
4
2

10

20

30

40

0
0

10

20

30

40

(h) Fourier Magnitude Distribution of LA Using PCA Bases Modulated by Illumination Spectra from (a) to (f)Figure 2. Various illumination spectra and their Fourier magnitude distribution. The first row shows the retrieved optimal illumination
spectrum (a), the spiky spectrum generated by ELS (b), the spectrum of the D2C lamp (c), the office fluorescent lamp spectrum (d), thecomposite spectrum from five color LEDs (e) and the spectrum of a 5000K white LED lamp (f). Their corresponding condition number ofM(l) is shown in the parentheses. The Fourier magnitude distributions of these six illumination spectra are shown in (g), while those of LA
are shown in (h).

To minimize the condition number of a linearly parameterized rectangular matrix is very challenging, since theobjective function is not differentiable, which would frustrate most derivative-based minimization algorithms. In thefollowing, we transform it into a bilinear matrix inequality(BMI) optimization problem. BMI has been extensively explored in control theory [1], for which there exist some complimentary off-the-shelf softwares, such as PENLAB [5].
Considering that a univariate quadratic function f (z) =z2 , z ≥ 0, is monotonically increasing, we minimize the following equivalent problem instead()
min cond2 (M(l)) = cond M(l)T M(l) .
(4)l

According to [2], in order to minimize the condition
number of the parametric positive definite matrix, Eqn. (4)can be rewritten into a fractional matrix inequality problem
min x, s.t., I1 ≼ y−1 M(l)T M(l) ≼ xI1 , y > 0,
x,y,l

(5)

in which I1 is the (u + v)-dimensional identity matrix.
Thanks to the Schur complement lemma, the following
equivalence holds][
1
xI1 M(l)T
T
≽ 0,
(6)
M(l) M(l) ≼ xI1 , y > 0 ⇔
M(l)
yI2
y
where I2 is the n-dimensional identity matrix.
Therefore, the condition number minimization problem
in Eqn.4 can be reformulated into the following bilinear ma-trix inequality problem][
xI1 M(l)T≽ 0, M(l)T M(l) ≽ yI1 , y > 0.
min x, s.t.,
M(l)
yI2
x,y,l
(7)
The bilinearity arises from the multiplication of M(l)Tand M(l), which is the only nonconvex constraint in Eqn.7.
This problem can be directly solved by using the PENLAB
software [5].
Using the 8-D PCA reflectance bases for A and 12-D
PCA fluorescence bases for B, PENLAB returns the retrieved optimal illumination spectrum in the 420nm-700nm
range, as shown in Fig.2(a). The corresponding condition number of M(l) is 1.8. Note that, we have run PENLAB 5000 times by starting from random initialization, andthis reported condition number is the smallest one. In thissense, we call the retrieved spectrum as the optimal spectrum. From Fig.2(a), we see that the optimal spectrum isvery spiky, and contains substantial high-frequency components, as revealed by its Fourier magnitude distribution in
the first subfigure of Fig.2(g).

3.3. Frequency Domain Analysis
Why are high-frequency components needed for our separation task? By considering the imaging equation in Eqn.3,
we have come up with an explanation from the viewpointof frequency modulation. Specifically, we can interpret theillumination spectrum l as a modulating signal of the reflectance bases A, whose spectral distribution has no effectat all on the fluorescent emission bases B, because of the3526

OPTIMAL
ELS
D2C
OFFICECOLOR LEDs
WHITE LEDICCV13(a) D2C(b) Ballast and Igniterintegration in Eqn.1. The condition number minimization
operation in Eqn.4 is to find a proper modulating signal
such that the correlation between LA and B is as small as
possible. When l contains high-frequency components, thefrequency of A can be lifted significantly, and thus the overlap of LA and B in the frequency-domain is greatly reduced.
This point can be easily observed by comparing the Fourier
magnitude distribution of LA in the first and last subfigureof Fig.2(h).
We have further analyzed the requirements on the illumination spectrum by means of frequency-domain signal processing. This theoretical analysis is presented in the supplementary material, since it is quite lengthy.

0.3
0.2
0.1

(c) Nikon ELS

Figure 3. The 6000K D2C HID lamp (a) and its driving accessories(b). The input voltage is 12V and the power is 55W. The whole
set costs about 30 USD, which is much more economical than the
programmable ELS (c).

0.4

0
0

OPTIMALELS
D2C
OFFICECOLOR LEDs
WHITE LEDICCV130.5

Evaluated RMSE

Evaluated RMSE

0.5

0.4
0.3
0.2
0.1

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Noise Levels (%)(a) Reflectance

0
0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Noise Levels (%)(b) Fluorescent Emission

Figure 4. Experiment results using synthetic data on the noise sensitivity of our proposed method under the OPTIMAL, ELS, D2C,
OFFICE, COLOR LEDs and WHITE LED illuminations shown in
Fig.2(a-f), as well as that of the ICCV13 method.

5000K white LED lamp in Fig.2(f), whose corresponding
condition number is 301.5.
We present the Fourier magnitude distribution of thesesix illumination spectra in Fig.2(g). We can observe that,similar to the optimal spectrum, both the spiky spectrum
by ELS and the D2C spectrum contain a large amount ofhigh-frequency components, which substantially lift the frequency of A, as shown in Fig.2(h). As expected, the white
LED spectrum has almost no high-frequency components,
and is inappropriate for the spectra separation task.

3.4. Illumination Spectra in Practice

3.5. Separation Algorithm

When a programmable light source is available, we cangenerate an illumination spectrum that approximates the numerically optimal spectrum very well. For example, by using a Nikon Equalized Lighting Source (ELS), we are able
to generate a spiky illumination spectrum with a condition
number of 4.9, as shown in Fig.2(b).
However, the irradiance intensity of this programmable
light source is quite weak, and as a consequence, the camera has to be slowed down so as to capture a bright enoughimage. This forces us to find a very strong lamp as the lightsource.
We have examined the spectra of tens of ordinary lamps
in daily life. Among them, we recognize that the D2C(6000K) HID lamp (see Fig.3) has very strong irradiancedue to its special lighting mechanism, and its spectrum ishighly spiky as shown in Fig.2(c). Its corresponding condition number is 12.4. This lamp is cheap and easily available,
since it is widely used as headlights in many brands of cars.
The office fluorescent lamp has a spiky spectrum as well, as
shown in Fig.2(d). Unfortunately, it has three spikes only,
and the resulting condition number is 55.2.
We have also tried to generate a spiky illumination spectrum by combining five narrow-band color LEDs, as shownin Fig.2(e). It leads to a condition number of 74.0. We believe that the primary reason lies in that the bandwidth of
the LEDs that we have is not narrow enough.
As comparison, we show the smooth spectrum of a

After the illumination spectrum l is determined, on thebasis of Eqn.3, we can easily separate the fluorescent and
[
]T [
]+
reflective components by α T β T = LA B i = M + i,+in which[ M ]denotes the Moore-Penrose pseudoinverse of
M = LA B .
When capturing a whole scene by using a hyperspectral
camera, we can separate all pixels simultaneously under theassumption that the illumination is uniform across the scene
by using the following linear operation][[]
α1 α2 · · · αm
= M + i1 i2 · · · im , (8)β1 β2 · · · βm
where m is the total number of pixels. Note that m can bevery large for a high-resolution hyperspectral image, however, this causes little difficulty, due to the fact that M + canbe precomputed offline and one only [needs to multiply] M +with the image measurement matrix i1 i2 · · · im . As
for the running time, our algorithm takes about 40 milliseconds in MATLAB (2012a) on a desktop with a 4-core i73770 CPU (3.4GHz) and 8GB memory, for a hyperspectral
cube with 57 bands and 640×512 spatial resolution.

3.6. Comparison with the State-of-the-Art [8]As mentioned in Sec.2, the method in [8] also uses highfrequency illuminations. Therefore, it would be interesting
to disclose the difference between our method and [8] in3527

more detail. The fundamental requirement in [8] is that theabsorption scalars under two different illuminations shouldbe the same. Specifically, given two illuminations l1 andl2 , we can obtain two equations on the basis of Eqn.2 asfollows)(∫l1 (λ̃)a(λ̃)dλ̃ e = L1 r + s1 e,
i1 = L1 r +(∫)(9)
i2 = L2 r +l2 (λ̃)a(λ̃)dλ̃ e = L2 r + s2 e.
To make sure that s1 equals s2 , a natural choice is to use
two high-frequency and complementary illumination spectra. What makes the illuminations difficult to generate is
the complementary restriction over the whole working spectral range, although a spectrum with high-frequency components is not uncommon. This might be the reason why [8]used a less widely available programmable lighting sourceas the illuminant.
In contrast, our method relies on the difference in fluorescent and reflective components, when responding to thefrequency modulation effect of illumination. The specificspectrum shape does not matter that much, as long as it
contains sufficient high-frequency components. This point
allows us to find an off-the-shelf lamp as the illuminant, asshown in Sec.3.4.

4. Experiments
4.1. Synthetic Data
We first evaluate the accuracy of our proposed method
under various illumination spectra by using synthetic data,and show how it compares with that of the state-of-the-art
method [8] using two high-frequency and complementary
sinusoid illuminations (referred to as ICCV13). For our separation algorithm, we evaluate all the six illumination spectra shown in Fig.2(a-f).
At each independent trial, we randomly select one spectrum from the 18 color patches on the Macbeth color
checker as reflectance, and randomly choose one fluorescent absorption-emission spectra pair from the fluorescent
spectra dataset [18]. To simulate image disturbance, Gaussian noise is added onto the synthesized hyperspectral signals, whose standard deviations vary from 0 to 5% relative
magnitude. At each noise level, we calculate the root meansquare error (RMSE) between the estimated spectra andtheir corresponding ground truth. According to the analysis in [8], we set the period of the high-frequency sinusoid
illuminations to be 40 nm.
Fig.4(a) and Fig.4(b) show the average RMSE over 100
independent trials for reflectance and fluorescent emission.
As expected, the theoretically optimal spectrum (OPTIMAL) makes the separation quite accurate and robust to
image noise. Although the ELS spectrum has almost the

1

1

0.5

0.5

0
420nm

490nm

560nm

630nm

0
700nm 420nm(a) Reflectance
1

0.5

0.5

490nm

560nm

630nm

0
700nm 420nm(c) Reflectance
1

0.5

0.5

490nm

560nm

630nm

0
700nm 420nm(e) Reflectance
1

0.5

0.5

490nm

560nm

630nm

(g) Reflectance

700nm

490nm

560nm

630nm

700nm

490nm

560nm

630nm

700nm

(f) Emission

1

0
420nm

630nm

(d) Emission

1

0
420nm

560nm

(b) Emission

1

0
420nm

490nm

0
700nm 420nm

490nm

560nm

630nm

700nm

(h) Emission

Figure 5. Comparison of the estimated spectra of four color fluorescent patches from our method under the D2C and ELS illuminations, as well as those from the ICCV13 method.

same effect as OPTIMAL, its real applications are still limited, due to the weak irradiance intensity of the programming light source. The separation accuracy of our selected
D2C illumination is sufficiently close to that of OPTIMAL
and ELS. The results from using the office fluorescent lamp
spectrum (OFFICE), the composite spectrum from multiple
color LEDs (COLOR LEDs) and the white LED spectrum(WHITE LED) are not accurate, since their corresponding
condition numbers are large.

4.2. Real Images
We use an EBA Japan NH-7 hyperspectral camera tocapture images, whose spatial resolution is 1280×1024 pixels. It has 57 bands from 420nm to 700nm, with an interval of 5nm. We use the Nikon ELS to generate highfrequency sinusoid illumination spectra for the ICCV13
method, whose period is 40 nm. Note that, since the irradiance intensity of the ELS is weak, the capture speed under
our selected D2C lamp is more than 100 times faster thanunder the ELS.
We first measure the reflectance and fluorescent emission
spectra of the four fluorescent patches in the right most col3528

(a) Input(g) Blue(b) Ref (Ours)(c) Fluo (Ours)(d) Ref (ICCV13)(h) Relighted w/ Fluo (i) Relighted w/o Fluo(j) Green(e) Fluo (ICCV13)(f) UV(k) Relighted w/ Fluo (l) Relighted w/o Fluo

Figure 6. Fluorescence and reflectance spectra separation of the plastic roller scene. From left to right, the first row shows the scene under
the D2C illuminant, the separated reflective and fluorescent components from our method and those from the ICCV13 method using highfrequency sinusoid illuminations. The scene under a UV lamp is shown at the right most for visual comparison. The second row shows theoriginal image and its relighted image with/without fluorescence under blue and green lights.

(a) Input(g) Blue(b) Ref (Ours)(c) Fluo (Ours)(d) Ref (ICCV13)(h) Relighted w/ Fluo (i) Relighted w/o Fluo(j) Green(e) Fluo (ICCV13)(f) UV(k) Relighted w/ Fluo (l) Relighted w/o Fluo

Figure 7. Fluorescence and reflectance spectra separation of the mesh-like pad scene and relighting under novel color illuminations. The
subfigures are arranged in the same way as that of Fig.6.

umn of Fig.5, which are used as ground truth. Fig.5 shows
the comparison between the ground truth and the estimatedspectra from our method under ELS and D2C, as well as
the results from ICCV13. From Fig.5, we observe that our
method under the ordinary D2C illuminant is comparable tothe state-of-the-art method in [8].
We have also used our method under the D2C lamp toseparate some other scenes with fluorescence, such as theplastic roller scene in Fig.6 and the mesh-like pad scene in
Fig.7. Both our method and the ICCV13 method achievesatisfactory separation results for these two scenes withstrong fluorescence. The cloth scene in Fig.8 contains weak
blue fluorescence in the white area. It is a common practicein the printing and painting industry to add some blue fluorescence for whitening. Although the ICCV13 method itself can handle scenes with blue fluorescence, the used ELS
illuminant contains almost no UV irradiance, and the blue
fluorescence is thus rarely excited. Our D2C lamp has some
irradiance in the UV range, and is able to excite blue fluorescence. This explains why our separated results are better
than those of ICCV13 in Fig.8. We believe that the capabil-

ity of our method to handle scenes with weak fluorescene
in the blue range would significantly widen the application
scope of hyperspectral fluorescence imaging.
One important application of the separated fluorescenceand reflectance spectra is to relight the scene under novel
color illuminations. To this end, we further estimate the absorption spectra from the separated fluorescent componentê in Eqn.2, by using the sparse dictionary method in [8].
The second row of Fig.6 and Fig.7 shows the captured andrelighted scenes under blue and green illuminations. When
fluorescence is considered, color discrepancy between therelighted scene and the captured one can be drastically alleviated. This justifies the necessity of taking fluorescence
into consideration for fluorescent-reflective scenes.
Our selected strong illuminant and fast separation algorithm pave the way for hyperspectral fluorescence imagingof dynamic scenes. Unfortunately, due to the speed limitation of our current hyperspectral camera, we are not able tocapture real-time hyperspectral video. In spite of that, wehave tried to demonstrate this potential by slowly moving
scene objects, and obtained some preliminary results for an3529

(a) Input(b) Ref (Ours)(c) Fluo (Ours)(d) Ref (ICCV13)(e) Fluo (ICCV13)(f) UV

Figure 8. Fluorescence and reflectance spectra separation of the cloth scene with weak green and blue fluorescence.

image sequence of a tennis ball, which are provided in the
supplementary materials.

5. Conclusions and Discussions
We have explored the requirements for illumination
spectra so as to enable the challenging task of separating
fluorescence and reflectance by using a single hyperspectral image. Guided by our numerically optimal design ofthe illumination spectrum, we have successfully found an
off-the-shelf and high-intensity lamp as the light source.
Our proposed separation algorithm is simple to implement
and fast enough to handle high-resolution hyperspectral images. Experiments have verified the accuracy of our proposed method.
As shown in Fig.5, the estimated spectra from our
method tend to be oversmoothed, or to be slightly oscillating when the original spectral segment is flat. This is acommon problem when using linear basis expression for inverse inference with noise. To alleviate this issue by finding
an even better illuminant is left as future work.

References[1] J. V. Antwerp and R. Braatz. A tutorial on linear and bilinear
matrix inequalities. Journal of Process Control, 10(4):363–
385, 2000. 4[2] S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 1st edition, 2004. 4[3] C. Chane, A. Mansouri, F. Marzani, and F. Boochs. Integration of 3D and multispectral data for cultural heritage applications: survey and perspectives. Image and Vision Computing, 31(1):91–102, 2013. 1[4] D. Ferris, R. Lawhead, and E. Dickman. Multimodal hyperspectral imaging for the noninvasive diagnosis of cervicalneoplasia. Journal of Lower Genital Tract Disease, 5(2):65–
72, 2001. 1, 2
[5] J. Fiala, M. Kocvara, and M. Stingl. PENLAB: A MATLAB solver for nonlinear semidefinite optimization. arXiv
preprint arXiv:1311.5240, pages 1–25, 2013. 4[6] Y. Fu, A. Lam, Y. Kobashi, I. Sato, T. Okabe, and Y. Sato.
Reflectance and fluorescent spectra recovery based on fluorescent chromaticity invariance under varying illumination.
IEEE Conference on Computer Vision and Pattern Recognition, pages 2163–2170, 2014. 1, 2
[7] Y. Fu, A. Lam, Y. Matsushita, I. Sato, and Y. Sato. Interreflection removal using fluorescence. European Conference
on Computer Vision, pages 203–217, 2014. 2[8] Y. Fu, A. Lam, I. Sato, T. Okabe, and Y. Sato. Separatingreflective and fluorescent components using high frequency

illumination in the spectral domain. IEEE International Conference on Computer Vision, pages 457–464, 2013. 1, 2, 3,
5, 6, 7[9] E. Fuchs. Separating the fluorescence and reflectance components of coral spectra. Applied Optics, 40(21):3614–3621,
2001. 1, 2
[10] S. Han, Y. Matsushita, I. Sato, T. Okabe, and Y. Sato. Camera spectral sensitivity estimation from a single image under unknown illumination by using fluorescence. IEEE Conference on Computer Vision and Pattern Recognition, pages
805–812, 2012. 2[11] S. Han, I. Sato, T. Okabe, and Y. Sato. Fast spectral reflectance recovery using dlp projector. Asian Conference on
Computer Vision, pages 323–335, 2010. 1[12] M. Hullin, M. Fuchs, I. Ihrke, H.-P. Seidel, and H. Lensch.
Fluorescent immersion range scanning. ACM Trans. Graph.,27(3):87:1–10, 2008. 2[13] G. Johnson and M. Fairchild. Full-spectral color calculations
in realistic image synthesis. IEEE Computer Graphics andApplications, 19(4):47–53, 1999. 1, 2
[14] S. Kim, S. Zhuo, F. Deng, C. Fu, and M. Brown. Interactive visualization of hyperspectral images of historical documents. IEEE Trans. Visualization and Computer Graphics,
16(6):1441–1448, 2010. 1[15] A. Lam and I. Sato. Spectral modeling and relighting ofreflective-fluorescent scenes. IEEE Conference on Computer
Vision and Pattern Recognition, pages 1452–1459, 2013. 1,2, 3[16] J. Leland, N. Johnson, and A. Arecchi. Principles of bispectral fluorescence colorimetry. Proc. SPIE, 3140:76–87,
1997. 2[17] L. Maloney. Evaluation of linear models of surface spectralreflectance with small numbers of parameters. J. Opt. Soc.
Am. A, 3(10):1673–1683, 1986. 3[18] G. McNamara, A. Gupta, J. Reynaert, T. Coates, andC. Boswell. Spectral imaging microscopy web sites and data.
Cytometry Part A, 69(8):863–871, 2006. 3, 6
[19] J. Park, M. Lee, M. Grossberg, and S. Nayar. Multispectral
imaging using multiplexed illumination. IEEE International
Conference on Computer Vision, pages 1–8, 2007. 1[20] J. Parkkinen, J. Hallikainen, and T. Jaaskelainen. Characteristic spectra of munsell colors. J. Opt. Soc. Am. A, 6(2):318–
322, 1989. 3[21] I. Sato, T. Okabe, and Y. Sato. Bispectral photometric stereo
based on fluorescence. IEEE Conference on Computer Vision
and Pattern Recognition, pages 270–277, 2012. 2[22] J. Suo, L. Bian, F. Chen, and Q. Dai. Bispectral coding:compressive and high-quality acquisition of fluorescence andreflectance. Optics Express, 22(2):1697–1712, 2014. 1, 23530

[23] S. Tominaga, T. Horiuchi, and T. Kamiyama. Spectral estimation of fluorescent objects using visible lights and an
imaging device. Proc. of the Color and Imaging Conference,
pages 352–356, 2011. 1, 2
[24] T. Treibitz, Z. Murez, B. Mitchell, and D. Kriegman. Shapefrom fluorescence. European Conference on Computer Vision, pages 292–306, 2012. 2[25] C. Zhang and I. Sato. Separating reflective and fluorescentcomponents of an image. IEEE Conference on Computer
Vision and Pattern Recognition, pages 185–192, 2011. 2, 3
[26] Y. Zheng, I. Sato, and Y. Sato. Spectra estimation of fluorescent and reflective scenes by using ordinary illuminants.
European Conference on Computer Vision, pages 188–202,
2014. 1, 33531