Per-Pixel Water and Oil Detection
on Surfaces with Unknown Reﬂectance
Chao Wang

Takahiro Okabe

Department of Artiﬁcial Intelligence
Kyushu Institute of Technology
Iizuka, Japanc wang@pluto.ai.kyutech.ac.jp

Department of Artiﬁcial Intelligence
Kyushu Institute of Technology
Iizuka, Japanokabe@ai.kyutech.ac.jp

Abstract—Water and oil detection is important for machine
vision applications such as visual inspection and robot motion
planning. It is known that water absorbs near infrared lightand oil absorbs near ultraviolet and blue light. Therefore,
observing at the absorbed wavelengths, the apparent spectral
reﬂectances of surfaces with water/oil are smaller than thatwithout water/oil. We could detect water/oil based on the above
absorption features by using a hyperspectral image, if the original
spectral reﬂectances of surfaces are known. However, in general,
the spectral reﬂectances of surfaces are unknown and spatiallyvarying. In this paper, we propose a novel per-pixel water andoil detection method based on the Lambert-Beer’s law and a
low-dimensional linear model for spectral reﬂectance. We showthat our method enables us to pixelwisely detect water and oil on
surfaces with unknown and spatially-varying reﬂectance at highaccuracy by using a hyperspectral image. The effectiveness of our
proposed method is conﬁrmed through a number of experiments
using real hyperspectral images.
Index Terms—water and oil detection, absorption, hyperspectral imaging, spectral reﬂectanceI. I NTRODUCTION
Detecting water and oil on object surfaces such as roadsurfaces, ﬂoors, and manufactured products in a non-contact
and non-destructive manner is vital to applications for autonomous driving systems, robot path planning [1], and visualinspection [2], [3]. In spite of its importance, water and oil
detection has received little attention in image processing
literature.
Image-based water and oil detection is a challenging problem. As shown in Fig.1 (a) and (b), water/oil is transparent/translucent for visible wavelengths, and then the appearance of water/oil on object surfaces signiﬁcantly depends
not only on water/oil themselves but also on the backgroundobject surface. Therefore, it is difﬁcult to distinguish surfaces
with/without water/oil from ordinary color images.
Interestingly, it is known that water absorbs near infrared(NIR) light, and oil absorbs near ultraviolet (NUV) and blue
light. Therefore, observing at those wavelengths, the brightnessof an object surface on which water/oil is present is smaller
than that without water/oil, because NIR/NUV light, incoming
to and outgoing from the object surface, is absorbed bywater/oil. In other words, the apparent spectral reﬂectance,
i.e. the ratio between the spectral radiance observed at asurface point and the spectral irradiance, is smaller when

ISBN: 978-9-0827-9706-0

Fig. 1. (a) The scene of an object surface with water (the reﬂectances areconverted into an RGB image). The ﬁrst column is pixels with water, and thesecond column is pixels without water. (b) The apparent reﬂectances (shown
by RGB image) of a white background with oil with different thicknesses from0.1cm to 1cm. (c) and (d) are two cases of water and oil detection. One isthat the surface with known spectral reﬂectance as shown in (c), and anotheris that the surface with unknown and spatially varying spectral reﬂectance(for example, three points on the surface with different spectral reﬂectance)
as shown in (d). The black lines are the surface original reﬂectance, the bluelines are the surface with water apparent reﬂectance, and the red lines are thesurface with oil apparent reﬂectance.

water/oil is present. Hence, we could detect water/oil by using
hyperspectral image if the original spectral reﬂectance ofobject surfaces is known (as shown in Fig.1 (c)).
However, the original spectral reﬂectances of object surfaces are unknown and spatially varying in general. Therefore,
when the apparent spectral reﬂectance of object surfaces issmall, we cannot distinguish two possible cases: one is thatthe original spectral reﬂectance itself is small, and anotheris that the apparent spectral reﬂectance is small due to theabsorption by water/oil (as shown in Fig.1 (d)).
Accordingly, in this paper, we propose a method for
per-pixel water and oil detection via hyperspectral imaging. Speciﬁcally, based on the Lambert-Beer’s law [4], ourproposed method estimates the original spectral reﬂectanceand liquid thickness at each surface point from its apparent
spectral reﬂectance simultaneously. First, we make use of
the low-dimensional linear combination model for spectral
reﬂectance from visible to NIR wavelengths. Then, water andoil are detected on the basis of the loss between the apparent
spectral reﬂectance computed by using the estimated spectral
reﬂectance, absorption coefﬁcient, and estimated liquid thick-601

EUSIPCO 2021

ness, and the observed apparent reﬂectance. To demonstrate
the effectiveness of our method, we conducted a numberof experiments using real images. We also compared theperformance of the low-dimensional linear model for spectral
reﬂectance reconstruction with sparse representation [5].
In summary, our main contributions are twofold. First, we
propose a novel method to simultaneously estimates the original spectral reﬂectance and liquid thickness at each surface
point simultaneously from a single hyperspectral image. Our
method enables us to detect water and oil on surfaces withunknown and spatially-varying reﬂectance. To our knowledge,
ours is the ﬁrst water and oil detection on object surfaces
with unknown spectral reﬂectance. Second, we compared thelow-dimensional linear model for spectral reﬂectance reconstruction with sparse representation to show which model ismore suitable for our water and oil detection method.
II. R ELATED W ORK
Water Detection: In general, static water has sky reﬂection
outdoors, and then the area of water with sky reﬂection canbe easily distinguished from other terrains by brightness [6],[7]. These aforementioned approaches are all promising but
the scene in which the above method can be applied is limited. Especially, surfaces with unknown and spatially-varying
reﬂectance, its brightness also spatially-varying, thus difﬁcult
to detect water. In recent years, using LDA [8] for classiﬁc
different colors of water and different backgrounds. Nevertheless, it affected by the liquid thickness leads to obtaining
undesired results. In contrast to the above methods, our method
enables us to detect water and oil with unknown thicknesses
on surfaces with unknown and spatially-varying reﬂectance.
Wang et al. [9] estimate the original NIR reﬂectance from thevisible reﬂectance to detect water. The difﬁculty of water andoil detection is that we cannot estimate the original spectral
reﬂectance from the reﬂectance in visible range because oil
absorbs a part of visible light. Our proposed method canestimate the physical parameters such as optical path length
as well as the presence/absence of water/oil.
Oil Detection: In remote sensing literature, Salem et al. [10]proposed an oil spill detection method by using hyperspectral
images in the early years. In recent years, Pabon et al. [11]have proposed a method based on diffuse reﬂection to detect
oil-contaminated soil. Menezes et al [12] and Liu et al. [13]have proposed methods to detect different types of oil spills.
Achard et al. [14] use spectral image unmixing in order toextract training samples and then apply SVM to detect onshorehydrocarbon. On the other hand, in food safety literature,
PCA is used to analyze spectral data to identify whether
oil is adulterated [15]. However, the above methods assume
known background (sea etc.) and require special experimental
equipments (test tube etc.). Therefore, they cannot detect oil(and water) on surfaces with unknown and spatially-varying
reﬂectance without using special equipments.
Lambert Beer’s Law: Attenuation due to light absorption is
described by the Lambert-Beer’s law [4] and has been used forwater depth estimation [16] and powder detection [17]. Asanoet al. [16] proposed a method of estimating the depth of objects
in water based on the multi-wavelength image using the factthat water absorbs NIR light. Zhi et al. [17] proposed a method
for powder recognition using multi-spectral imaging. They use
the lambert-Beer’s law to modeling powders. In contrast to
this approach, our method uses the Lambert-Beer’s law toestimates the original spectral reﬂectance and liquid thickness
at each surface point from its apparent spectral reﬂectance.
Low-Dimensional Linear Model for Spectral Reﬂectance
Reconstruction: Parkkinen et al. [18] measured and analyzed
the spectral reﬂectance of the Munsell Color Book and showedthat the visible spectral reﬂectance in the visible light can beapproximated by a linear combination of a small number ofbasis vectors. Wang et al. [9] extend Parkkinen et al. methodfrom visible wavelengths to NIR wavelengths. In recent years,
the low-dimensional linear model has been widely used toreconstruct the spectral reﬂectance [19]–[23]. Compared withthe above method, we apply it to the detection of water andoil per pixel.
III. P ROPOSED M ETHOD
A. Apparent Spectral Reﬂectance
We capture an image of a scene of interest by using a
hyperspectral camera. Suppose that the spectral irradiance ofthe scene is uniform and known up to per-pixel scale, i.e. thescene is illuminated by a single light source or multiple lightsources with the same SPD (spectral power distribution). The
per-pixel spectral irradiance can be measured up to a scaleby taking a hyperspectral image of the scene and the whitebalance and diffuse reﬂectance target of Spectralon at the sametime for example.
We denote the spectral radiance and irradiance at a certain
surface point by S(λ) and L(λ) respectively. Then, we express
the reﬂectance at the surface point based on the LambertBeer’s law [4]. Given wavelength λ, the Lambert-Beer law
accurately expresses light absorption as the relation between
radiance S(λ) and irradiance L(λ),S(λ) = ρ(λ)L(λ)e−α(λ)l(1)

in which ρ(λ) represents the original spectral reﬂectance ofthe surface, l represents liquid thickness (optical path length),and α(λ) denotes the spectral absorption coefﬁcient.
Therefore, the apparent spectral reﬂectance ρ̂(λ) at thesurface point is given by1ρ̂(λ) =S(λ)= ρ(λ)e−α(λ)l .
L(λ)

(2)

The apparent spectral reﬂectance is determined by the surface
original spectral reﬂectance, spectral absorption coefﬁcient,and liquid thickness.
1 We determine the scale of the apparent spectral reﬂectance so that its
maximum value with respect to λ is 1 without loss of generality.

602

Fig. 2. The experimental results : (a) Pseudo RGB image, (b) 400nm image, and (c) 975nm image of the scene where water and oil are present. (d) Correctlabel: with water (blue), with oil (yellow), background (black). Detection results: (e) RGB L1sparse, (f) RGB PCA, (g) HSI L1sparse, (h) HSI proposed (ourproposed method)

B. Low-Dimensional Linear Model for Spectral Reﬂectance
Parkkinen et al. [18] studied the spectral reﬂectances ofthe Munsell Color Book chips, and showed that the spectral
reﬂectances in visible wavelengths are approximately represented by linear combinations of the basis functions derived
via Principal Component Analysis (PCA) [24]. Speciﬁcally, aspectral reﬂectance ρ(λ) is represented as
ρ(λ) ≃

N∑cn bn (λ),

(3)

n=1

where λ stands for the wavelength of incoming and outgoinglight, and bn (λ) and cn are the n-th basis function (n =1, 2, 3, ..., N ) and its coefﬁcient respectively.
Recently, Wang et al. [9] experimentally show that the lowdimensional linear model in eq.(3) is applicable not only tovisible wavelengths but also to visible to NIR wavelengths.
In this paper, our proposed method uses the low-dimensional
linear model from visible to NIR wavelengths.

the spectral reﬂectance is non-negative. Then, our methodminimizes the squared error between ρ̃(λ) and ρ̂(λ) as∫min(ρ̃(λ) − ρ̂(λ))2 dλ.
(5){cn ,α(λ),l}Speciﬁcally, we estimate the liquid type (water or oil ornone) and the liquid thickness discretely, and estimate thesurface spectral reﬂectance coefﬁcient continuously. We ﬁx theliquid types (water or oil or none) and the liquid thickness,solve the spectral reﬂectance by the constrained least square
method. And perform the above calculation while changing theliquid types (water or oil or none), and the liquid thickness. In
particular, the absorption coefﬁcient α(λ) is from three types
of water, oil, and none, and the thickness l is in the range of
0 mm to 10 mm, and the step size is 0.1 mm.
In addition, as a comparison method, we compare the effectof sparse representation [5] in detecting water and oil. We
use Munsell Color Book chips reﬂectances as a dictionary for
representing spectral reﬂectance. Here, the apparent spectral
reﬂectance can be approximated bymin

C. Per-Pixel Water and Oil Detection{W,α(λ),l}We assume that the spectral absorption coefﬁcients of waterand oil are known, and the absorption coefﬁcient of none is
0. Our proposed method detects water and oil per pixel by
estimating the liquid types (water or oil or none), the original
spectral reﬂectance and liquid thickness at each surface point
simultaneously from its apparent spectral reﬂectance. Specifically, substituting Eq.(3) into Eq.(2), the apparent spectral
reﬂectance can be approximated by
[N
]
∑
(4)
ρ̃(λ) =
cn bn (λ) e−α(λ)l .
n=1Here, ρ̃(λ)is approximated apparent spectral reﬂectance. The
coupling coefﬁcient cn is estimated for each pixel so that||ρ̃(λ) − W De−α(λ)l ||2 + k||W ||1 ,(6)

where we deﬁne the sparse coefﬁcient matrix as W , the color
chips dictionary matrix as D, and k as a sparse parameter.
IV. E XPERIMENTS
A. Water and Oil Detection
In the experiment, object surfaces were illuminated by ahalogen lamp, and a hyperspectral camera manufactured by
EBA JAPAN was used. We sampled spectral reﬂectances from
380 to 1000 nm at 5-nm intervals, and then obtained 125
bands of hyperspectral images. In this paper, the spectral
irradiance of the scene is measured by using a standard
diffuse reﬂector. Then, the apparent spectral reﬂectances wereobtained by dividing the spectral radiances by the measured
spectral irradiance.

603

420nm

460nm

500nm

540nm

580nm

620nm

660nm

800nm

970nm

(a)
(b)
Water (c)
(d)
(e)
(a)
(b)
Oil

(c)
(d)
(e)

Fig. 3. (a) The original reﬂectance of the surface. (b) The apparent spectral reﬂectance of the liquid (water and oil) on the surface. (c) The reconstructed
reﬂectance of the proposed method. (d) The RMSE between the reconstructed reﬂectance and the original reﬂectance. (e) RMSE between reconstructed
reﬂectance and apparent spectral reﬂectance.

We tested the performance of water and oil detection on
17 surfaces; 4 cloths with complex texture, 3 ﬂoor tiles withuniform colors, 3 pieces of paper with different colors, 6 piecesof leather with different colors, 1 wood, and 1 mixed scene(consisting of 5 different materials with 16 different colors). In
order to obtain the ground truth labels for quantitative analysis,
we ﬁrst captured the image of the object surface withoutwater/oil, and then captured the image of the object surface
with water/oil, and applied the active contour method [25] totheir subtraction. The following comparisons are conducted
to show the effectiveness of the hyperspectral image and thelow-dimensional linear model.
RGB L1sparse: We use the RGB apparent reﬂectancegenerated by assuming Canon EOS-1D Mark III’s spectral
sensitivity [26] as the input data, and approximate the RGB
apparent reﬂectance based on the sparse representation [5].
Here, we set the range of the sparse parameter k in eq.(6) tobe between 1 and 10, and select the optimal value that yields
the largest mFscore as a comparison for our method.
RGB PCA: We use the RGB apparent reﬂectance generated
by assuming Canon EOS-1D Mark III’s spectral sensitivity [26] as the input data, and approximate the RGB apparent
reﬂectance based on the low-dimensional linear model with
PCA [24]. Here, the number of basis functions is 2.
HSI L1sparse: We use the apparent spectral reﬂectancecaptured by a hyperspectral camera as the input image, andwe approximate the apparent spectral reﬂectance based on thesparse representation [5]. Here, we set the range of the sparse
parameter k in eq.(6) to be between 1 and 10, and select the
optimal value that yields the largest mFscore as a comparison

TABLE IR ESULTS FOR METHOD PERFORMANCE .

MethodRGB L1sparse
RGB PCA
HSI L1sparse
HSI proposed

mPrecision
0.12880.10500.86200.9706mRecall
0.45490.34750.9326
0.9768mFscore0.20080.16120.89600.9737for our method.
First, in Fig. 2(a)-(d), we show are the Pseudo RGB image,the 400nm image, the 975nm image of the scene where waterand oil are present, and the label, from left to right. We cansee that water and oil detection based on the RGB image or
the absorption band image is difﬁcult.
Second, the results of RGB L1sparse and RGB PCA are
shown in Fig. 2(e)(f) show that RGB images do not work
well no matter whether the low-dimensional linear model andsparse representation, effective results cannot be obtained. Thisis because water is transparent under visible light and theapparent spectral reﬂectance is the same as surfaces spectral
reﬂectance. In addition, the experimental result shows that itis difﬁcult to detect oil when using RGB images because theabsorption of blue wavelengths is small.
Third, in Fig. 2(g), we show the sparse representation
result, which uses a small number of spectral reﬂectance toapproximate the apparent spectral reﬂectance. In Fig. 2(h),compared with the above methods, our proposed method has
obtained robust results.
Table I shows the results of quantitative evaluation. Threeindexes, mRecall, mPrecision and mFscore, were used for604

quantitative analysis. They represent average recall, average
precision, and average Fscores for water, oil, and backgrounddetection. The mFscores of RGB L1sparse and RGB PCA
methods are very low. In addition, in the HSI L1sparse, theapparent spectral reﬂectance is approximated based on a small
number of Munsell Color Book data. If the sparse parameterk is small, the result will be affected by noise, otherwise, itwill reduce the data used and make approximation difﬁcult.
Therefore, we chose the k that obtains the best mFsorce as acomparison. The experimental result shows that our method
performs better than the above methods.
B. Spectral Reconstruction
Our water and oil detection method is based on the estimation of the liquid thickness and surface spectral reﬂectancefrom a hyperspectral image. It is very interesting that we haveobtained the reconstruction of surface spectral reﬂectance as
an intermediate product while detecting water and oil.
The results are shown in Fig. 3. We choose the root meansquare error (RMSE) as our evaluation metric. We calculate the
RMSE between the reconstructed spectral reﬂectance and thesurface original spectral reﬂectance as shown in Fig. 3(d) and
the RMSE between the reconstructed spectral reﬂectance andthe surface apparent spectral reﬂectance as shown in Fig. 3(e).
The result show that our spectral reconstruction is effective.
In particular, the absorption wavelength result (framed in red
in the Fig. 3), shows that the surface texture under the liquid
is clearly reconstructed.
Because we can estimate the liquid type and the surface
reﬂectance reconstruction, we can edit the apparent spectral
reﬂectance by changing any element in the eq.(2). In addition,
it is also very helpful for surface classiﬁcation applications
when we do not know whether there is liquid on the surface.
V. C ONCLUSION AND F UTURE W ORK
In this paper, we proposed a method for per-pixel water andoil detection on surfaces with unknown and spatially-varying
reﬂectance. Based on the Lambert-Beer s law and a lowdimensional linear model for spectral reﬂectance, water andoil are detected on the basis of the loss between the apparent
spectral reﬂectance computed by using the estimated spectral
reﬂectance, absorption coefﬁcient, estimated liquid thickness,and the observed apparent reﬂectance. We conducted a numberof experiments using real images, and conﬁrmed that our
proposed method performs well. In the future, we will reduce
the number of bands to design a simple camera for water andoil detection.
ACKNOWLEDGMENT
This work was supported by JSPS KAKENHI Grant Numbers JP17H01766 and JP20H00612.
R EFERENCES[1] B. Ichter, J. Harrison, and M. Pavone, “Learning sampling distributions
for robot motion planning,” in Proc. IEEE ICRA 2018, 2018, pp. 7087–
7094.

[2] F. Pernkopf and P. O’Leary, “Image acquisition techniques for automatic
visual inspection of metallic surfaces,” NDT & E International, Vol. 36,
No. 8, pp. 609–617, 2003.
[3] S. Agnisarman, S. Lopes, K. Madathil, K. Piratla, and A. Gramopadhye, “A survey of automation-enabled human-in-the-loop systems forinfrastructure visual inspection,” Automation in Construction, Vol. 97,
pp. 52–76, 2019.
[4] A. Beer and P. Beer, “Determination of the absorption of red light incolored liquids,” Annalen der Physik und Chemie, Vol. 86, No. 5, pp.
78–88, 1852.
[5] R. Tibshirani, “Regression shrinkage and selection via the lasso,”
Journal of the Royal Statistical Society: Series B (Methodological), Vol.
58, No. 1, pp. 267–288, 1996.
[6] A. Rankin and L. Matthies, “Daytime water detection based on color
variation,” in Proc. IEEE/RSJ IROS 2010, 2010, pp. 215–221.
[7] A. Rankin, L. Matthies, and P. Bellutta, “Daytime water detection basedon sky reﬂections,” in Proc. IEEE ICRA 2011, 2011, pp. 5329–5336.
[8] A. Fisher, N. Flood, and T. Danaher, “Comparing landsat water index
methods for automated water classiﬁcation in eastern australia,” Remote
Sensing of Environment, Vol. 175, pp. 167–182, 2016.
[9] C. Wang, M. Okuyama, R. Matsuoka, and T. Okabe, “Per-pixel water
detection on surfaces with unknown reﬂectance,” to appear in the IEICE
Transactions on Information and Systems.
[10] F. Salem, M. Kafatos, T. El-Ghazawi, R. Gomez, and R. Yang, “Hyperspectral image analysis for oil spill detection,” in Summaries ofNASA/JPL Airborne Earth Science Workshop, Pasadena, CA, 2001, pp.
5–9.
[11] R. Pabón, C. De Souza Filho, and W. De Oliveira, “Reﬂectance andimaging spectroscopy applied to detection of petroleum hydrocarbon
pollution in bare soils,” Science of the Total Environment, Vol. 649, pp.
1224–1236, 2019.
[12] J. Menezes and N. Poojary, “A fusion approach to classify hyperspectral
oil spill data,” Multimedia Tools and Applications, Vol. 79, No. 7, pp.
5399–5418, 2020.
[13] B. Liu, Y. Li, Q. Zhang, and L. Han, “Assessing sensitivity ofhyperspectral sensor to detect oils with sea ice,” Journal of Spectroscopy,Vol. 2016, 2016.
[14] V. Achard, S. Fabre, A. Alakian, D. Dubucq, and P. Déliot, “Direct
or indirect on-shore hydrocarbon detection methods applied to hyperspectral data in tropical area,” in Earth Resources and Environmental
Remote Sensing/GIS Applications IX. International Society for Optics
and Photonics, 2018, Vol. 10790, p. 107900N.
[15] N. Abu-Khalaf and M. Hmidat, “Visible/near infrared (vis/nir) spectroscopy as an optical sensor for evaluating olive oil quality,” Computers
and Electronics in Agriculture, Vol. 173, pp. 105445, 2020.
[16] Y. Asano, Y. Zheng, K. Nishino, and I. Sato, “Shape from water:
Bispectral light absorption for depth recovery,” in Proc. ECCV 2016.
Springer, 2016, pp. 635–649.
[17] T. Zhi, B. Pires, M. Hebert, and S. Narasimhan, “Multispectral imaging
for ﬁne-grained recognition of powders on complex backgrounds,” inProc. IEEE CVPR 2019, 2019, pp. 8699–8708.
[18] J. Parkkinen, J. Hallikainen, and T. Jaaskelainen, “Characteristic spectraof munsell colors,” JOSA A, Vol. 6, No. 2, pp. 318–322, 1989.
[19] A. Lam, A. Subpa-Asa, I. Sato, T. Okabe, and I. Sato, “Spectral imaging
using basis lights.,” in Proc. BMVC 2013, 2013.
[20] S. Han, I. Sato, T. Okabe, and Y. Sato, “Fast spectral reﬂectance recovery
using dlp projector,” IJCV, Vol. 110, No. 2, pp. 172–184, 2014.
[21] M. Kitahara, T.Okabe, C.Fuchs, and H.Lensch, “Simultaneous estimation of spectral reﬂectance and normal from a small number of images,”in Proc. VISAPP (1) 2015, 2015, pp. 303–313.
[22] Ying Fu, Antony Lam, Imari Sato, and Yoichi Sato, “Adaptive spatialspectral dictionary learning for hyperspectral image restoration,” IJCV,Vol. 122, No. 2, pp. 228–245, 2017.
[23] H. Hidaka, Y. Monno, and M. Okutomi, “Spectral reﬂectance estimation
using projector with unknown spectral power distribution,” in Proc. CIC
2020, 2020.
[24] S. Wold, K. Esbensen, and P. Geladi, “Principal component analysis,”
Chemometrics and Intelligent Laboratory Systems, Vol. 2, No. 1-3, pp.
37–52, 1987.
[25] T. Chan and L. Vese, “Active contours without edges,” IEEE TIP, Vol.
10, No. 2, pp. 266–277, 2001.
[26] J. Jiang, D. Liu, J. Gu, and S. Süsstrunk, “What is the space of spectral
sensitivity functions for digital color cameras?,” in Proc. IEEE WACV
2013, 2013, pp. 168–179.

605