Image-based Material Editing for Making Reflective Objects Fluorescent
Daichi Hidaka and Takahiro Okabe
Department of Artificial Intelligence, Kyushu Institute of Technology, Japanokabe@ai.kyutech.ac.jp

Keywords:

Image-based Material Editing, Photometric Consistency, Fluorescence, Reflection, Spectral Irradiance.

Abstract:

Fluorescent materials give us a unique sense of quality such as self-luminous ones, because they absorb lightwith certain wavelengths and then emit light with longer wavelengths. The existing methods for image-basedmaterial editing make objects in an image specular, translucent, and transparent, but they do not address
fluorescent materials. In this paper, we propose a method for making reflective objects in a single input image
fluorescent by adding photorealistic fluorescent components to the objects of interest. Specifically, we showthat photometrically consistent fluorescent components can approximately be represented by using the 3-band(RGB) spectral irradiance on the surface of a reflective object, and then compute the fluorescent components
on the basis of intrinsic image decomposition without explicitly estimating the object’s shape and the lightsources illuminating it from the input image. We conducted a number of experiments using both synthetic andreal images, and confirmed that our proposed method is effective for making reflective objects fluorescent.

1

INTRODUCTION

Fluorescence is a very common phenomenon observed both in natural objects such as minerals andplants and in man-made objects such as papers andclothes (Barnard, 1999). Fluorescent materials absorb light with certain wavelengths and then emit lightwith longer wavelengths, in contrast to reflective materials which reflect light with the same wavelengths
as those of the incident light. Therefore, they give usa unique sense of quality such as self-luminous materials.
Image-based material editing is a technique for
automatically and photo-realistically replacing one
material in an image with another. Khan et al. (Khanet al., 2006) propose a method for making an object ina single input image specular, translucent, and transparent. Since the appearance of an object depends on
the shape of the object and the light sources illuminating it, their proposed method roughly estimates thedepth map of the object and the illumination distribution of a scene from the input image on the basis ofheuristics. Liu et al. (Liu et al., 2017) propose an endto-end network architecture that predicts the shape,
illumination, and material from a single image, andmake use of the predicted results for material editingof diffuse and specular objects.
On the other hand, fluorescent materials are notaddressed in the context of image-based material editing. They are addressed mainly in the context of re-flection separation, i.e. separating reflective and fluorescent components under narrow-band camera assumption (Zhang and Sato, 2013), under narrow-bandillumination assumption (Koyamatsu et al., 2019),
and by using high-frequency illumination in the spectral domain (Fu et al., 2016b), and in the context ofimage-based modeling, i.e. recovering the spectral
properties of reflection and fluorescence (Fu et al.,2016a).
In this paper, we propose a method for making reflective objects in a single input image fluorescent byadding photorealistic fluorescent components to theobjects of interest. Specifically, we show that photometrically consistent fluorescent components canapproximately be represented by using the 3-band(RGB) spectral irradiance on the surface of a reflective object. Therefore, our proposed method computes the fluorescent components on the basis of intrinsic image decomposition (Tappen et al., 2005)
without explicitly estimating the object’s shape andthe light sources illuminating it from the input image.
We conducted a number of experiments using bothsynthetic and real images, and confirmed that the approximation represented by using the 3-band spectral
irradiance works well and that our proposed method
is effective for making reflective objects fluorescent.
The application scenarios of our method include theappearance simulation of fluorescent materials for
traffic signs, commercial posters, manufactured products, and so on.
355

Hidaka, D. and Okabe, T.
Image-based Material Editing for Making Reflective Objects Fluorescent.
In Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020) - Volume 1: GRAPP, pages
355-360
ISBN: 978-989-758-402-2
Copyright © 2020 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved

GRAPP 2020 - 15th International Conference on Computer Graphics Theory and Applications

The main contributions of this study are twofold.
First, based on the fluorescent model, we show thatphotometrically consistent fluorescent components
can approximately be represented by using the 3-band
spectral irradiance on the surface of a reflective object. It enables us to compute the fluorescent components on the basis of intrinsic image decomposition without explicitly estimating the object’s shapeand the light sources illuminating it from an input image. Second, we experimentally confirmed that our
proposed method is effective for making reflective
objects in a single input image fluorescent. Specifically, we confirmed that the approximation represented by using the 3-band spectral irradiance works
well through qualitative and quantitative evaluation
using synthetic images, and that our method provides
visual enrichment of photographs with virtual fluorescence.

2

PROPOSED METHOD

2.1

Overview

For fluorescent materials, the pixel value ib (xx) at apoint x on an object surface consists of a reflective
component rb (xx) and a fluorescent component fb (xx)asib (xx) = rb (xx) + fb (xx),(1)
where b (b = 1, 2, 3) stands for the band (RGB) of acolor camera. Our proposed method assumes that an
object of interest has only the first term, i.e. the reflective component, and then adds the second term,
i.e. the fluorescent component to the object surface.
In this section, we explain the fluorescence modeland the diffuse reflection model, and then explain our
method for computing photometrically consistent fluorescent components.

2.2

Fluorescence Model

In general, a pure fluorescent material absorbs lightwith a certain wavelength λ0 , and then emits lightwith a longer wavelength λ00 . Those properties aredescribed by the absorption spectrum a(λ0 ) and theemission spectrum e(λ00 ) (Zhang and Sato, 2013).
When a fluorescent material is illuminated by distant light sources from various directions with various
wavelengths, the fluorescent component is given by
Z

Z
0
0
>
ω, λ )ω
ω n (xx)dωω dλ0
fb (xx) =
a(λ )
l(ω
×
356

Z

cb (λ00 )e(λ00 )dλ00 .

(2)

ω, λ0 ) describes the energy of the light rays
Here, l(ωfrom the direction ω and with the wavelength λ0 , i.e.
the spectral illumination distribution of a scene. n (xx)and cb (λ00 ) are the surface normal at the point x andthe b-th band’s spectral sensitivity of a camera respectively. It is reported that the brightness of a fluorescentcomponent approximately obeys the Lambert model,
i.e. is represented by the inner product between thesurface normal and the light source direction (Satoet al., 2012; Treibitz et al., 2012).
We assume that the spectral sensitivity of a camera
is narrow-band as is often assumed in color constancy
algorithms. Namely, the spectral sensitivity is approximately represented by using the Dirac delta function;cb (λ00 ) ' δ(λ00 − λb ). Then, the integral with respect
to λ00 in eq.(2) is represented asZ

cb (λ00 )e(λ00 )dλ00 ' e(λb ),(3)where we assume the peak value of the spectral sensitivity is 1 for the sake of simplicity.
The integral with respect to ω in eq.(2) is the spectral irradiance s(xx, λ0 ) due to incident light rays fromvarious directions to the point x with the wavelength
λ0 , and is defined bys(xx, λ0 ) ≡Zω, λ0 )ωω> n (xx)dω
ω.
l(ω

(4)

We substitute eq.(4) into the integral with respect to λ0
in eq.(2), and then approximately represent the integral by the summation with respect to the wavelengthsof the 3 bands asZa(λ0 )s(xx, λ0 )dλ0 ' ∑ a(λb )s(xx, λb ).

(5)b

Substituting eq.(3) and eq.(5) into eq.(2), we obtainfb (xx) ' e(λb ) ∑ a(λb0 )s(xx, λb0 ).

(6)

b0

Therefore, the brightness of the fluorescent component is computed by the sum of the products between
the 3-band absorption spectrum a(λb0 ) and the 3-band
spectral irradiance s(xx, λb0 ). The color of the fluorescent component is determined by the 3-band emission
spectrum e(λb ).

2.3

Diffuse Reflection Model

According to the Lambert model, the diffuse reflection component is described as
Z

Z
0
0
0
>
ω, λ )ω
ω n (xx)dωω dλ0
rb (xx) =cb (λ )ρ(xx, λ ) l(ω=Zcb (λ0 )ρ(xx, λ0 )s(xx, λ0 )dλ0(7)Image-based Material Editing for Making Reflective Objects Fluorescent

1
0.8
0.6
0.4
0.2

400

500

600

700

0
400

(a)

500

600

(c)

(d)

(e)

(f)

(g)

(b)

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

(a)

0
400

500

600

700

400

(c)

500

600

700

(d)

Figure 1: The spectral data for synthetic images. (a) the
spectral intensity of a light source, (b) the spectral sensitivity of a camera, (c) the spectral reflectances of a reflective
object (red: left, blue: right), and (d) the absorption andemission spectra of fluorescent materials (red: upper, green:lower).

Here, ρ(xx, λ0 ) is the spectral reflectance at the point x ,and we use the definition of the spectral irradiance ineq.(4).
Assuming a narrow-band camera in a similar manner to the previous subsection, the diffuse reflection
component is given byrb (xx) ' ρ(xx, λb )s(xx, λb ).

(8)

Therefore, the diffuse reflection component is computed by the products between the 3-band spectral reflectance ρ(xx, λb ) and the 3-band spectral irradiances(xx, λb ).

2.4

(b)
700

Adding Fluorescence

Our proposed method adds fluorescent components,
whose absorption spectrum a(λ0 ) and emission spectrum e(λ00 ) are specified by a user 1 , to an object surface of interest in a single input image. Since boththe 3-band absorption spectrum a(λb ) and the 3-band
emission spectrum e(λ0b ) in eq.(6) are given, the problem of computing photorealistic fluorescent components results in the estimation of the 3-band spectral
irradiance s(xx, λ0b ).
On the other hand, the diffuse reflection components also depend on the 3-band spectral irradiance asshown in eq.(8), and therefore we can estimate the 3band spectral irradiance from them. Specifically, we
1 The absorption and emission spectra can be selectedfrom the McNamara dataset (McNamara et al., 2006).

Figure 2: The experimental results using synthetic images.
(a) the input image of a reflective object, (b) (d) (f) the fluorescent components, and (c) (e) (g) the result images, i.e.
the fluorescent components are added to and the reflective
components due to absorption are subtracted from the input
image. The spectral rendering, our proposed method, andthe naive method are used for (b) (c), (d) (e), and (f) (g)respectively.

make use of intrinsic image decomposition without
explicitly estimating the object’s shape and the lightsources illuminating it from the input image.
Thus, our proposed method computes the fluorescent components from the user-provided absorption and emission spectra and the 3-band spectral irradiance estimated via intrinsic image decomposition.
Specifically, our method replaces the pixel value ib (xx)in the input image with i0b (xx) computed asi0b (xx) = ib (xx) + fb (xx) − ρ(xx, λb )a(λb )s(xx, λb ).

(9)Here, the third term means that the reflective component decreases because a part of incident light rays isabsorbed by the fluorescent material.

3

EXPERIMENTS

3.1

Synthetic Images

To confirm the effectiveness of our proposed method,
we conducted qualitative and quantitative evaluationby using synthetic images. Specifically, we compared
the following three methods for replacing a reflective
object with a fluorescent one.
357

GRAPP 2020 - 15th International Conference on Computer Graphics Theory and Applications• Spectral Rendering: computes the fluorescentcomponents according to eq.(2) under the condition that all the spectral data, i.e. the absorption
spectrum a(λ), the emission spectrum e(λ), thespectral sensitivity cb (λ), and the spectral illumination distribution l(xx, λ) are given.
• Our Proposed Method: computes the fluorescent components according to eq.(6) under thecondition that the 3-band absorption spectruma(λb ), the 3-band emission spectrum e(λb ), andthe 3-band spectral irradiance s(xx, λb ) 2 are given.
• Naive Method: computes the fluorescent components according to eq.(6), but the pixel valuesof the input image are used instead of the 3-band
spectral irradiance without using intrinsic image
decomposition.
We synthesized the image of a sphere illuminated bya distant light source from the frontal direction. The
spectral data used for synthesizing the input image areshown in Figure 1: (a) the spectral intensity of a lightsource, (b) the spectral sensitivity of a camera, (c) the
spectral reflectances of a reflective object (red: left,
blue: right), and (d) the absorption and emission spectra of fluorescent materials (red: upper, green: lower).
Figure 2 shows (a) the input image of a reflective object, (b) (d) (f) the fluorescent components, and (c)(e) (g) the result images, i.e. the fluorescent components are added to and the reflective components
due to absorption are subtracted from the input image.
The spectral rendering, our proposed method, and thenaive method are used for (b) (c), (d) (e), and (f) (g)respectively.
First, we can see that the fluorescent components
and the result images computed by using (b) (c) the
spectral rendering and (d) (e) our proposed method
are similar to each other. The SSIM (Wang et al.,2003) and PSNR between (c) the result image ofthe spectral rendering and (e) that of our method are
0.971 and 40.41 respectively. Those results qualitatively and quantitatively show that our approximation
represented by using the 3-band spectral irradianceworks well.
Second, we can see that the fluorescent components and the result images computed by using (f)(g) the naive method are significantly different fromthose computed by using (b) (c) the spectral rendering. Specifically, (f) the fluorescent components arebrighter at the area with bluish reflectance. This is because we specified fluorescent materials which absorb
bluish light as shown in Figure 2 (d) and the naive
method uses not the 3-band spectral irradiance but the
2 In the experiments using synthetic images, we assume

that the 3-band spectral irradiance is known.

358

(a)

(b)

(c)

(d)

Figure 3: The experimental results using real images: an
object with uniform reflectance. (a) the input image, (b) the
3-band spectral irradiance, (c) the fluorescent components,and (d) the result image computed by using our proposed
method.

pixel values of the input image without using intrinsic
image decomposition. The SSIM and PSNR between(c) the result image of the spectral rendering and (g)that of the naive method are 0.791 and 20.64 respectively. Those results qualitatively and quantitatively
show that the use of the 3-band spectral irradianceis important for computing photorealistic fluorescentcomponents.

3.2

Real Images

To demonstrate the effectiveness of our proposed
method for real images, we tested our method on threedifferent conditions: (i) an object with uniform reflectance, (ii) an object with texture, and (iii) an object
illuminated by an additional light source. In our experiments, we used SIRFS (Barron and Malik, 2015)
for intrinsic image decomposition.
First, we tested an object with uniform reflectanceas shown in Figure 3. Figure 3 shows (a) the input
image, (b) the 3-band spectral irradiance, (c) the fluorescent components, and (d) the result image computed by using our proposed method. Here, we specified the 3-band absorption and emission spectra asa = (0, 0, 0.4)> and e = (0.8, 0.2, 0)> respectively. In
other words, we assume that a fluorescent material absorbs bluish light and emits reddish light. We can seethat (c) the fluorescent components and (d) the result
image computed by using our method on the basisof the 3-band spectral irradiance are photometricallyconsistent with the input image; the object is illuminated mainly from the upper-right direction.

Image-based Material Editing for Making Reflective Objects Fluorescent

(a)

(b)

(d)

(a)

(e)

(b)

(f)

(c)

(g)

(d)

(h)

(c)

(e)

Figure 4: The experimental results using real images: an
object with texture. (a) the input image, (b) the reflectanceof the object, (c) the 3-band spectral irradiance, (d) the fluorescent components, and (e) the result image computed byusing our proposed method.

Second, we tested an object with texture as shownin Figure 4. Figure 4 shows (a) the input image,(b) the reflectance of the object, (c) the 3-band spectral irradiance, (d) the fluorescent components, and(e) the result image computed by using our proposed
method. The specified 3-band absorption and emission spectra are the same as the above. We can seethat (d) the fluorescent components and (e) the result image computed by using our method are photometrically consistent with the input image; the object
is illuminated almost uniform ambient light. Since
our method makes use of the 3-band spectral irradiance estimated via intrinsic image decomposition, thecomputed fluorescent components are not contaminated by the texture on the object surface.
Third, we tested an object illuminated by an additional light source as shown in Figure 5. Figure 5
shows (a) the input image of an object illuminatedby ambient light and (e) that illuminated by ambient
light and an additional bluish light source from right.
(b)/(f), (c)/(g), and (d)/(h) are the 3-band spectral irradiance, the fluorescent components, and the result images computed from (a)/(e) the corresponding input
images by using our proposed method. The specified
3-band absorption and emission spectra are the sameas the above. We can observe the photometricallyconsistent effects of the additional light source in (f),(g), and (h). Specifically, the 3-band spectral irradi-

Figure 5: The experimental results using real images: anadditional light source. (a) the input image of an object illuminated by ambient light and (e) that illuminated by ambient light and an additional bluish light source from right.
(b)/(f), (c)/(g), and (d)/(h) are the 3-band spectral irradiance,
the fluorescent components, and the result images computedfrom (a)/(e) the corresponding input images by using our
proposed method.

ance becomes bluish due to the bluish additional lightsource, and therefore the fluorescent components at
the right areas become brighter since the specified fluorescent material absorbs bluish light.

4

CONCLUSION

In this paper, we proposed a method for making reflective objects in a single input image fluorescent byadding photorealistic fluorescent components to theobjects of interest. Specifically, we show that photometrically consistent fluorescent components can approximately be represented by using the 3-band spectral irradiance on the surface of a reflective object, andthen compute the fluorescent components on the ba359

GRAPP 2020 - 15th International Conference on Computer Graphics Theory and Applications

sis of intrinsic image decomposition without explicitly estimating the object’s shape and the light sources
illuminating it from the input image. We conducted
a number of experiments using both synthetic andreal images, and confirmed that our proposed method
is effective for making reflective objects fluorescent.
Taking specular reflection and inter-reflection components in an input image into consideration is one of
the future directions of this study.

ACKNOWLEDGMENTS
This work was partially supported by JSPS KAKENHI Grant Numbers JP18H05011 and JP17H01766.

REFERENCES
Barnard, K. (1999). Color constancy with fluorescent surfaces. In Proc. CIC1999, pages 257–261.
Barron, J. and Malik, J. (2015). Shape, illumination, andreflectance from shading. IEEE TPAMI, 38(7):1670–
1687.
Fu, Y., Lam, A., Sato, I., Okabe, T., and Sato, Y. (2016a).
Reflectance and fluorescence spectral recovery via actively lit RGB images. IEEE TPAMI, 38(7):1313–
1326.
Fu, Y., Lam, A., Sato, I., Okabe, T., and Sato, Y. (2016b).
Separating reflective and fluorescent components using high frequency illumination in the spectral domain. IEEE TPAMI, 38(5):965–978.
Khan, E., Reinhard, E., Fleming, R., and Bülthoff, H.
(2006). Image-based material editing. In Proc. ACM
SIGGRAPH2006, pages 654–663.
Koyamatsu, K., Hidaka, D., Okabe, T., and Lensch, H.
(2019). Reflective and fluorescent separation undernarrow-band illumination. In Proc. IEEE CVPR2019,
pages 7577–7585.
Liu, G., Ceylan, D., Yumer, E., Yang, J., and Lien, J.M. (2017). Material editing using a physically based
rendering network. In Proc. IEEE ICCV2017, pages
2261–2269.
McNamara, G., Gupta, A., Reynaert, J., Coates, T., andBoswell, C. (2006). Spectral imaging microscopy web
sites and data. Cytometry Part A, 69A(8):863–871.
Sato, I., Okabe, T., and Sato, Y. (2012). Bispectral photometric stereo based on fluorescence. In Proc. IEEE
CVPR2012, pages 270–277.
Tappen, M., Freeman, W., and Adelson, E. (2005). Recovering intrinsic images from a single image. IEEE
TPAMI, 27(9):1459–1472.
Treibitz, T., Murez, Z., Mitchell, G., and Kriegman,
D. (2012). Shape from fluorescence. In Proc.
ECCV2012, pages 292–306.
Wang, Z., Bovik, A., Sheikh, H., and Simoncelli, E. (2003).
Image quality assessment: from error visibility to
structural similarity. IEEE TIP, 13(4):600–612.

360

Zhang, C. and Sato, I. (2013). Image-based separationof reflective and fluorescent components using illumination variant and invariant color. IEEE TPAMI,
35(12):2866–2877.