VOL. E104-D NO. 10
OCTOBER 2021
The usage of this PDF file must comply with the IEICE Provisions
on Copyright.
The author(s) can distribute this PDF file for research andeducational (nonprofit) purposes only.
Distribution by anyone other than the author(s) is prohibited.

IEICE TRANS. INF. & SYST., VOL.E104–D, NO.10 OCTOBER 2021

1555

PAPER

Special Section on Picture Coding and Image Media Processing

Per-Pixel Water Detection on Surfaces with Unknown Reflectance
Chao WANG†a) , Student Member, Michihiko OKUYAMA† , Nonmember, Ryo MATSUOKA†† ,and Takahiro OKABE† , Members

SUMMARY Water detection is important for machine vision applications such as visual inspection and robot motion planning. In this paper, we
propose an approach to per-pixel water detection on unknown surfaces witha hyperspectral image. Our proposed method is based on the water spectral
characteristics: water is transparent for visible light but translucent/opaquefor near-infrared light and therefore the apparent near-infrared spectral reflectance of a surface is smaller than the original one when water is present
on it. Specifically, we use a linear combination of a small number of basis vector to approximate the spectral reflectance and estimate the originalnear-infrared reflectance from the visible reflectance (which does not depend on the presence or absence of water) to detect water. We conducted anumber of experiments using real images and show that our method, which
estimates near-infrared spectral reflectance based on the visible spectral reflectance, has better performance than existing techniques.
key words: water detection, absorption, hyperspectral imaging, NIR, spectral reflectance1.

Introduction

Detecting water on object surfaces such as road surfaces,
floors, and manufactured products is important for various
applications such as autonomous driving systems, robot path
planning [1], [2], and visual inspection [3], [4]. In spite ofits importance, water detection has received little attention
in computer vision literature.
Image-based water detection is a challenging problem.
This is because water is transparent for visible wavelengths,
and then the appearance of water on an object surface significantly depends not on water itself but on the backgroundobject surface. Therefore, it is diﬃcult to distinguish surfaces with/without water from ordinary color images.
Interestingly, it is known that water is translucent or
opaque for near infrared (NIR) wavelengths, because water
absorbs NIR light [5]. Therefore, observing at NIR wavelengths, the brightness of an object surface on which water
is present is smaller than that without water, because NIR
light, incoming to and outgoing from the object surface,
is absorbed by water. In other words, the apparent spectral reflectance, i.e. the ratio between the spectral radianceobserved at a surface point and the spectral irradiance, isManuscript received January 10, 2021.
Manuscript revised April 26, 2021.
Manuscript publicized June 6, 2021.
†
The authors are with Department of Artificial Intelligence,
Kyushu Institute of Technology, Iizuka-shi, 820–8502 Japan.
††
The author is with Faculty of Environmental Engineering, The
University of Kitakyushu, Kitakyushu-shi, 808–0135 Japan.
a) E-mail: c wang@pluto.ai.kyutech.ac.jp
DOI: 10.1587/transinf.2021PCP0002

Fig. 1 The apparent spectral reflectance at a surface point when water isabsent (red line) and present (blue line) in it.

smaller when water is present (see Fig. 1). Hence, we could
detect water if the original spectral reflectance of an object
surface without water is known, and if the apparent spectralreflectance of the point is smaller than the original one for
NIR wavelengths.
However, the original spectral reflectances of object
surfaces are unknown and spatially varying in general.
Therefore, when the apparent spectral reflectance of an object surface is small, we cannot distinguish two cases: one isthat the original spectral reflectance itself is small, and another is that the apparent spectral reflectance is small due tothe absorption by water.
Accordingly, in this paper, we propose a method for
per-pixel water detection via hyperspectral imaging. Specifically, our proposed method estimates the original spectralreflectance at each surface point in NIR wavelengths fromits apparent spectral reflectance in visible wavelengths, thatare invariant to the presence/absence of water. To this end,
we make use of the low-dimensional linear combination
model for spectral reflectance from visible to NIR wavelengths. Then, water is detected on the basis of the diﬀerence between the estimated original spectral reflectance andthe apparent one in NIR wavelengths. To demonstrate theeﬀectiveness of our method, we conducted a number of experiments using real images, and then show that our method
works better than existing methods.
The main contributions of this study are twofold. First,
we propose a method for per-pixel water detection via hyperspectral imaging on the basis of wavelength-dependentc 2021 The Institute of Electronics, Information and Communication Engineers
Copyright ⃝IEICE TRANS. INF. & SYST., VOL.E104–D, NO.10 OCTOBER 2021

1556

opacity of water. Our method enables us to detect water on surfaces with unknown and spatially-varying reflectance. Second, we experimentally show that the spectral reflectances from visible to NIR wavelengths are approximately represented by using the low-dimensional linear combination model. The spectral reflectance estimatedon the basis of the low-dimensional model would be usefulfor other applications such as spectral rendering and material classification.
2.

Related Work

Water Detection: In remote sensing literature, there are
well-known methods such as NDWI, HDWI, and their extension methods [6], [7]. These methods focus on detecting water bodies such as rivers, ponds, and oceans. On theother hand, we focus on detecting water on surfaces where
the reflectance is unknown and spatially-varying. Therefore,
the purpose of our method is diﬀerent from these methods. In computer vision literature, in general, static water
has sky reflection outdoors, and then the area of water withsky reflection can be easily distinguished from other terrain
by brightness [8], [9]. However, outdoor water detection isstill very challenging when sky reflection is not observed. If
there is water on an object surface, the ratio of the spectral
radiances on the surface at the two specific NIR bands willchange greatly when the lighting is constant [10]. However,the spectral reflectance of surfaces such as factory automation is unknown and non-uniform, which makes it more difficult to identify water using only NIR spectral reflectances.
Water is the most complicated case. Because the appearanceof water on an object’s surface significantly depends not on
the water itself but on the background object surface. Despite this variability, water abosrbs light, diﬀerent colors ofwater and diﬀerent backgrounds are classified by LDA [11].
However, it may be aﬀected by the depth of the water toobtain undesired results. In contrast to the above methods,
our method pays more attention to the property that water
is transparent in visible wavelengths and translucent/opaquein the NIR wavelengths.
Hyperspectral Characteristics of Water: Shimano etal. [12], [13] proposed a method for wetness estimation on
the basis of spectral analysis. Specifically, by analyzing thescattering of light inside the wet object, the change in the
spectral brightness in the visible wavelengths due to water
wetting is explained. Asano et al. [14] proposed a method of
estimating the depth of objects in water on the basis of themulti-wavelength images using the fact that water absorbs
NIR light. When light travels in water, water rarely absorbs
visible light, which explains why water appears transparent to human eyes. In contrast, water absorbs NIR lightfrom 830 nm to 1000 nm. In contrast to this approach, ourmethod focuses not only on the visible spectral reflectanceor the NIR spectral reflectance but on the connections between them.
Hyperspectral Object Detection: Hyperspectral imagery has been used in reconnaissance and surveillance ap-

plications where targets of interest are detected and identified [15]. The orthogonal subspace projection (OSP) algorithm [16] maximizes the signal-to-noise ratio (SNR) in
the subspace orthogonal to the background subspace. However, this method requires background information beforehand, which is generally not achievable. The automatic target generation process (ATGP) [17] generates a set of targets from image data in an unsupervised manner which willsubsequently be classified. It can be used to search for a
specific target in unknown scenes. Compared with the classical detection technology, we propose a novel method for
estimating NIR reflectance from visible reflectance, whichsolves the problem of detecting water and background when
the background is complex and unknown. Chakrabarti andZickler [18] use hyperspectral images of indoor and outdoor
scenes, derive an optimized spatio-spectral basis for representing hyperspectral image patches using PCA, and explore
statistical models for the coeﬃcients in this basis. We proposed a method for estimating a NIR spectral reflectancefrom an apparent spectral reflectance in a visible light range
on the basis of the spectral correlation between visible and
NIR wavelengths.
3. Proposed Method
3.1

OutlineIn this Section, we explain the details of our proposed
method. First, we describe a low-dimensional linear modelof spectral reflectance from visible to NIR wavelengths.
Second, we explain how to estimate the original spectral reflectance in NIR wavelengths from the apparent spectral reflectance in visible wavelengths under the assumption thatthe spectral irradiance of a scene is known. Third, we
describe per-pixel water detection on the basis of the difference between the apparent and estimated original spectral reflectances in NIR wavelengths. Figure 2 shows the
flowchart of our method.
3.2

Low-Dimensional Linear Model

Parkkinen et al. [19] studied the spectral reflectances of the
Munsell Color Book chips, and showed that the spectral reflectances in visible wavelengths are approximately represented by linear combinations of the basis functions derived
via Principal Component Analysis (PCA). Specifically, aspectral reflectance ρ(λ) is represented asN∑cn bn (λ),(1)
ρ(λ) ≃
n=1

where λ stands for the wavelength of incoming and outgoing light, and bn (λ) and cn are the n-th basis function(n = 1, 2, 3, ..., N) and its coeﬃcient respectively.
In contrast to Parkkinen et al., we study the spectralreflectances of the Munsell Color Book chips not only invisible wavelengths but also in NIR wavelengths. We experimentally show that the low-dimensional linear model in

WANG et al.: PER-PIXEL WATER DETECTION ON SURFACES WITH UNKNOWN REFLECTANCE

1557

Fig. 2

The flowchart of our proposed method for per-pixel water detection.

Eq. (1) is applicable not only to visible wavelengths but also
to visible to NIR wavelengths.
3.3 Estimation of NIR Spectral Reflectance
We capture an image of a scene of interest by using a hyperspectral camera. Suppose that the spectral irradiance ofthe scene is uniform and known up to per-pixel scale, i.e.
the scene is illuminated by a single light source or multiple
light sources with the same spectral intensity. The per-pixel
spectral irradiance can be measured up to a scale by taking a
hyperspectral image of the scene and the white balance anddiﬀuse reflectance target of Spectralon at the same time forexample.
We denote the spectral radiance and irradiance at a certain surface point by s(λ) and l(λ) respectively. Then, the apparent spectral reflectance ρ̂(λ) at the surface point is given
by†
s(λ)
.
(2)
ρ̂(λ) =l(λ)
We assume that surfaces of interest are opaque andwater-repellent, and that water is clear, containing no scattering medium, and thin. Therefore, we consider water astransparent liquid for visible wavelengths. Since water istransparent for visible light, the apparent spectral reflectanceis the same as the original spectral reflectance in visible
wavelengths independent of the presence/absence of water.
†
We determine the scale of the apparent spectral reflectanceso that its maximum value with respect to λ is 1 without loss ofgenerality.

Then, we estimate the original spectral reflectance in NIR
wavelengths from the apparent spectral reflectance in visible wavelengths. Specifically, on the basis of Eq. (1), we
compute the coeﬃcient cn by solving the set of linear equations:N∑
cn bn (λm ), (m = 1, 2, 3, ..., M).
(3)
ρ̂(λm ) ≃n=1

Here, λm is the visible wavelength at which the apparent
spectral reflectance is given, and we set M ≥ N. Substituting the computed coeﬃcient cn into Eq. (1), we obtain theoriginal spectral reflectance ρ(λ).
3.4Per-Pixel Water Detection

Our proposed method detects water per pixel by exploiting
the observation that the apparent spectral reflectance at asurface point is smaller than the original one in NIR wavelengths when water is present on the surface point. Specifically, we tested two measures. The first measure Q1 isthe sum of the diﬀerence between the estimated spectral reflectance ρ(λ) and the apparent spectral reflectance ρ̂(λ) in
NIR wavelengths [λl , λh ]:
λh
∑
Q1 =
[ρ(λ) − ρ̂(λ)].
(4)
λ=λl

The second one Q2 is the maximum value of the diﬀerence
between the estimated spectral reflectance ρ(λ) and the apparent spectral reflectance ρ̂(λ) in NIR wavelengths:
Q2 = max [ρ(λ) − ρ̂(λ)].
(5)λ=[λl ,λh ]IEICE TRANS. INF. & SYST., VOL.E104–D, NO.10 OCTOBER 2021

1558

Since ρ(λ) ≃ ρ̂(λ) when water is absent, both theQ1 and Q2 take values near 0. On the other hand, sinceρ(λ) > ρ̂(λ) when water is present, both the Q1 and Q2
take positive values. Thus, our method detect water by using
thresholding Q1 or Q2 . In practice, due to the approximation
errors in the low-dimensional linear model and the noises inhyperspectral images, both the Q1 and Q2 distribute around0 and some positive values when water is absent and present
respectively. Therefore, we evaluate the approximation errors of the spectral reflectances of the Munsell Color Bookchips and some labeled samples, and then perform Receiver
Operator Characteristic (ROC) analysis for determining thethreshold [20]. In Sect. 4, we show that the measure basedon the maximum diﬀerence Q2 performs better than thestraightforward measure Q1 in terms of F-score.
4.

Experiments

4.1 Low-Dimensional Linear Model Results
In this paper, the statistical structure of spectral reflectanceis analyzed, and a low-dimensional linear combination
model for the spectral reflectances from visible to NIR
wavelengths is described. Similar to Parkkinen et al. [19],our analysis also based on the principal component analysis.
We measured total of 1302 spectral reflectances from visible to NIR wavelengths from the Munsell Color Book chips
with diﬀerent hues, saturations, and values. In our proposed
method, the first 10 principal components are used for reconstructing the measured spectral reflectance.
First, we obtained the basis vectors of the spectral reflectance from the visible to NIR wavelengths by using principal component analysis, and show the first five basis vectors in Fig. 3 (a). The first to fifth basis vectors are displayedin order of decreasing eigenvalues. The first eigenvector corresponds to the mean of the measured vectors, and its flatness indicates uniform covering of the color space. The cumulative contributions of eigenvalues are 0.9967 and 0.9997
for the first five and ten eigenvalues respectively.
In the second phase, we tested the reconstruction of
the spectral reflectance by the basis vectors. In order toverify the feasibility of the linear approximation of thespectral reflectance from visible to NIR wavelengths, wehave imported the real-world hyperspectral image dataset
TokyoTech-dataset [21] (color images are shown in Fig. 3(b). The dataset contains 59-band hyperspectral images, including both visible and NIR spectral reflectance, of colorful
objects with rich textures. Therefore, it is useful for the evaluation of various hyperspectral imaging and image restoration tasks. Figure 3 (c) represents some results of restoring
the TokyoTech-dataset using our dataset. The dotted lines
are the actual spectral reflectances and the solid lines are
the their approximations by using the low-dimensional linear model. We can see that the restoration results are verysatisfactory; the RMSE error per band per pixel is 0.0183. It
shows that the spectra reflectances of natural scenes can bereconstructed almost exactly by using 10 basis vectors.

4.2

Per-Pixel Water Detection Results

In the experiment, object surfaces were illuminated by astandard halogen lamp, and a hyperspectral camera manufactured by EBA JAPAN was used. We sampled spectralreflectances from 380 to 1000 nm at 5-nm intervals, andthen obtained 125 bands of hyperspectral images. In thispaper, we assume that object surfaces is illuminated by alight source of known spectral intensity. So, the spectral irradiance of the light source is measured by using a standard
diﬀuse reflector. Then, the input spectral reflectances wereobtained by dividing the spectral radiances by the measured
spectral intensity of the light source. The process and resultsof the proposed method are shown in Fig. 4 (c-e).
First, we captured the hyperspectral images of scenes
as with 125 bands. To facilitate understanding, we show thecolor images of the scenes by converting the hyperspectral
images into the RGB images in Fig. 4(a).
Second, for each pixel, the apparent spectral reflectance in the visible wavelengths is used to estimate theoriginal spectral reflectance in NIR wavelengths from 835
nm to 1000 nm (34bands). In this paper, we used 10 basis vectors, and the coeﬃcients were computed from theapparent spectral reflectances between 380 nm to 830 nm(91bands).The apparent and estimated spectral reflectances
at surface points without and with water are shown in Fig. 4(c) and (d) respectively. Here, the solid red and blue lines
are for the apparent spectral reflectances, and the black dotted lines are for the estimated original spectral reflectances.
As shown in Fig. 4(c)(d), water is transparent for visible
wavelengths, and then the appearance of water on an object surface significantly depends not on the water itself but
on the background object surface. The apparent spectralreflectance and original spectral reflectance are almost thesame for visible wavelengths. On the other hand, in NIR
wavelengths, the brightness of an object surface on whichwater is present is smaller than that without water, because
NIR light, incoming to and outgoing from the object surface, is absorbed by water. In other words, the apparent
spectral reflectance is smaller when water is present. In NIR
wavelengths, as shown in Fig. 4(c), the apparent spectral reflectance without water is almost the same as the estimatedoriginal spectral reflectance. However, as shown in Fig. 4(d), the apparent spectral reflectance is smaller than the estimated original spectral reflectance.
Finally, (e) the histogram of the diﬀerence measure
Q1 or Q2 over the entire image is made on the basis of
Eq. (3). The optimal threshold for classifying surface points
with/without water is determined by ROC analysis withsome samples with the ground truth labels. We used thethreshold to perform binary classification on a test scene.
4.3

Comparison with Existing Methods

We tested the performance of water detection on unknown
and complex backgrounds using complex textured cloths

WANG et al.: PER-PIXEL WATER DETECTION ON SURFACES WITH UNKNOWN REFLECTANCE

1559

Fig. 3 (a) Basis vector of spectral reflectances from visible to near-infrared. The first to fifth basis
vector are displayed in order of decreasing eigenvalues. (b) TokyoTech-dataset samples in color images.
(c) Restoration results (using our dataset for TokyoTech-dataset [21]). The RMSE per band per pixel is0.0183.

Fig. 4 The process and results of the method: (a) the color images of the targets converted from thehyperspectral images, (b) the results of water detection; white/black stand for with/without water, (c)the apparent spectral reflectance of a pixel in the image without water (solid red line), (d) the apparent
spectral reflectance of a pixel in the image with water (solid blue lines). The dotted black lines in (c)and (d) are the estimated original spectral reflectances, (e) the histogram of the diﬀerence between theestimated spectral reflectance and the apparent spectral reflectance.

((1)-(4)) and 6 diﬀerent colors of leather ((5)-(6)) as objects. Odd numbers are non-water scenes and even numbers
are water scenes. In order to obtain the ground truth labels
for quantitative analysis, we first captured the image of thesurface without water, and then captured the image of theobject surface with water, and made positive solution labels
by finding diﬀerences and active contour [22].
To confirm the eﬀectiveness of our proposed method,
we compared the performance of our method with those of

the following methods.
NIR975nm (method based on only one NIR band) :In the wavelengths of 380 to 1000 nm, we selected the wavelength of 975 nm at which the water absorption is maximum,
and then used the apparent spectral reflectance at that bandas input.
NIROnly (method based on ratio of the shortwave infrared bands) [10] : We used the ration of apparent spectral reflectances ar two diﬀerent wavelengths in

IEICE TRANS. INF. & SYST., VOL.E104–D, NO.10 OCTOBER 2021

1560

NIR, 975 nm/830 nm, at which water absorption is maximum/minimum.
DTDCA (method based on orthogonal subspace
projection) [17] : The relative water absorption is given as
the detection target, the background is estimated, and thescore of the detector is used as an input. This is a common
method, please refer to [17] for detail.
WI2015 (method based on linear discriminant analysis) [11] : We assume the water absorption is known. The
same statistical analysis was used to determine the coefficients that best separated the training class, like thosewhich minimize the within-class variance and maximize thebetween-class variance, using linear discriminant analysis
classification. The training pixels included images with water and without water. Then, we obtain the feature vector
and use the normalized value of the linear combination of
the recognition object and the feature vector as input. In particular, we use visible to NIR wavelengths instead of visible
to shortwave infrared wavelengths (in the existing method)
as objects.
NDWI [23] and HDWI [24] : NDWI and HDWI are
well-known indices for water detection. Since the spectral range of our hyperspectral camera is limited from 380
nm to 1000 nm, we define NDWI as NDWI = (Green −NIR)/(Green + NIR), where the Green and NIR bands
are [490,580]nm and [780,860]nm respectively. We define
HDWI as HDWI = (Red − NIR)/(Red + NIR), where the
Red and NIR bands are [650,700]nm and [705,850]nm respectively.
RXD (method based on Reed-XiaoLi detector) [25] :Algorithms commonly used in spectral anomaly detection.
please refer to [25] for details.
VSM (method based on vector-subspace model)[26] : The VSM method also makes use of the lowdimensional linear model in a similar manner to ours, but
the way of calculating the coeﬃcients is diﬀerent. The VSM
method uses the reflectance from visible to NIR wavelengths
instead of NIR wavelength in Eq. (3).
We compared the performance of our proposed methodwith those of closely related methods. The performanceswere evaluated by using the receiver operator characteristic (ROC) curves, which plot the true positive rate (TPR)
against the false positive rate (FPR) for a range of thresholdvalues [20]. TPR is the percentage of reference water pixels that are correctly classified, and FPR is the percentage ofreference non-water pixels incorrectly classified as water.
Diﬀerent methods have diﬀerent shaped ROC curves,
reflecting their variable accuracies in classifying the data
across diﬀerent thresholds (Fig. 5). Ours are the steepest(closer to the top left in Fig. 5), representing more accurate results than the other indexes across a limited rangeof thresholds. Across the full range of thresholds, Ours
achieves the highest area under the ROC curve (see AUC
in Table 1), as the index is more accurate when conservative
or liberal thresholds are used.
In addition, we also introduced recall, precision, andF-score for quantitative analysis. The results are shown

Fig. 5ods.

The ROC curves for our proposed method and the existing meth-

Table 1 The quantitative comparison with the closely related existing
methods in terms of Recall, Precision, F-score, and AUC.
MethodRecallPrecisionF-scoreAUC
Ours (Q2 )Ours (Q1 )NIR975nm
NIROnly [10]DTDCA [17]WI2015 [11]HDWI [24]
NDWI [23]RXD [25]
VSM [26]

0.91960.85590.74470.86990.55930.42870.25110.16010.26350.71190.90650.89340.50530.91900.63150.30340.09890.15760.07900.82680.91300.87420.60210.89380.59320.35530.14190.15880.12150.76510.9896
0.99390.96350.98280.94090.86890.54390.62610.65130.9521in Table 1. First, due to the influence of the near-infrared
light absorption of water and the reflectance of the surface
itself, the NIR method (NIR975nm) using only one bandhas a higher recall, but the precision is extremely low. A
method that takes account of the changes in the absorption
rate (NIROnly) reduces the recall rate and improves precision. This is because the diﬀerence in reflectance of the
background itself is not taken into account. This is why the
AUC and F-score of both methods are not so high. Second,in the method of simply estimating the reflectance of the surface of the object (DTDCA), it can be seen that comparedwith the above methods, the AUC is improved, but the Fscore is low. This is because the transparent color of water inthe visible range shows the background color, which resultsin large errors for backgrounds. Third, in the method of classifying water background and anhydrous background withLDA (WI2015), the results of water thickness and noisecannot be considered, so the method does not work well.
Fourth, Table 1 clearly shows that NDWI, HDWI, and RXD
methods are not suitable for detecting water on the surface
with spatially-varying reflectance. Fifth, VSM method is not
as good as our method. Because the VSM method uses thereflectance from visible to NIR wavelengths, and then thecoeﬃcients are contaminated by the apparent reflectance at
NIR wavelengths when water is present. This is a possible
reason for false negatives(Fig. 6 (l)-(6)). Finally, considering the relationship between visible light, NIR, and the spectral characteristics of water, the recall, and precision of ourWANG et al.: PER-PIXEL WATER DETECTION ON SURFACES WITH UNKNOWN REFLECTANCE

1561

Fig. 6 The experimental results : (a) The scene is shown in the RGB image (water is absent/present
in the scenes with odd/even numbers), (b) true labels, and the result images of methods: (c) NIR975nm,(d) NIROnly method, (e) DTDCA, (f) WI2015, (g) ours with Q1 , (h) ours with Q2 , (i) NDWI, (j) HDWI,(k) RXD, and (l) VSM.

method is a good balance. Our method also has a higher Fscore and AUC than existing methods. At the same time, we
can also see that our method (Q2 ) is better than our method(Q1 ) in terms of F-score. This may be caused by the the additive noises in the captured hyperspectral images, and theapproximation errors of the low-dimensional linear model.
In Fig. 6, we show (c-d) a method using only NIR, (e)the existing method based on background estimation, and(f) the existing method based on LDA classification. First,
Fig. 6 (c) shows that the NIR single-band method is diﬃcult
to classify the originally small reflectance in near-infrared
and the small reflectance due to water absorption. This is
the reason why some colors are detected as water. We cansee that in Fig. 6 (d), the results of using only NIR have false
negative. We can see that part of the water is considered to
be background (as shown in Fig. 6 (d)-(6)). This is caused by
the surface with spatially-varying reflectance. Our method
has false negative too due to specular reflection on the water surface. Ours may be solved by using a polarizing filter.
Second, in Fig. 6 (e), it can be seen that the overall detection result is not clearer than the NIR only method due tousing background estimating. Unfortunately, because water
is transparent for visible light, resulting in water with multiple visible range reflectances. This is why relying solely on
the NIR absorption of water as a target may result in incorrect estimates of the reflectance of background components.
Third, as shown in Fig. 6 (f), the results are not well due toerrors caused by diﬀerent water depths and noise. Finally,
Fig. 6 (h) clearly shows that our method (Q2 ) performs better than the above methods. At the same time, in Fig. 6(g),we can also see that our method (Q1 ) detects water when
there is no water in the background. There are two reasonsfor errors. The first reason is noises in the hyperspectral images. The second reason is the use of the low-dimensional
linear model; its approximation accuracy is limited when asmall number of basis functions is used.

5. Conclusion and Future WorkIn this paper, we proposed a method for per-pixel water detection on surfaces with unknown and spatially-varying reflectance. Our proposed method estimates the original spectral reflectance at each surface point in NIR wavelengths
from its apparent spectral reflectance in visible wavelengths.
Specifically, we make use of the low-dimensional linear
combination model for spectral reflectance from visible to
NIR wavelengths. Then, water is detected on the basis of
the diﬀerence between the estimated original spectral reflectance and the apparent one in NIR wavelengths. We conducted a number of experiments using real images, and confirmed that our proposed method performs better than theexisting technology. In the future, we will reduce the number of bands to design a simple camera for water detection.
Acknowledgments
This work was supported by JSPS KAKENHI Grant Numbers JP17H01766 and JP20H00612.
References[1] A. Khelloufi, N. Achour, R. Passama, and A. Cherubini, “Tentaclebased moving obstacle avoidance for omnidirectional robots withvisibility constraints,” 2017 IEEE/RSJ Int. Conf. Intelligent Robots
and Systems (IROS), pp.1331–1336, IEEE, 2017.
[2] B. Ichter, J. Harrison, and M. Pavone, “Learning sampling distributions for robot motion planning,” 2018 IEEE Int. Conf. Robot.
Autom. (ICRA), pp.7087–7094, IEEE, 2018.
[3] F. Pernkopf and P. O’Leary, “Image acquisition techniques for automatic visual inspection of metallic surfaces,” NDT & E International, vol.36, no.8, pp.609–617, Dec. 2003.
[4] S. Agnisarman, S. Lopes, K.C. Madathil, K. Piratla, and A.
Gramopadhye, “A survey of automation-enabled human-in-the-loop

IEICE TRANS. INF. & SYST., VOL.E104–D, NO.10 OCTOBER 2021

1562

systems for infrastructure visual inspection,” Automation in Construction, vol.97, pp.52–76, Jan. 2019.
[5] J.A. Curcio and C.C. Petty, “The near infrared absorption spectrumof liquid water,” JOSA, vol.41, no.5, pp.302–304, 1951.
[6] Y. Zhou, J. Dong, X. Xiao, T. Xiao, Z. Yang, G. Zhao, Z. Zou, andY. Qin, “Open surface water mapping algorithms: A comparison ofwater-related spectral indices and sensors,” Water, vol.9, no.4, p.256,
2017.
[7] T. Bijeesh and K. Narasimhamurthy, “Surface water detection anddelineation using remote sensing images: A review of methods andalgorithms,” Sustainable Water Resources Management, vol.6, no.4,
pp.1–23, 2020.
[8] A. Rankin and L. Matthies, “Daytime water detection based on color
variation,” 2010 IEEE/RSJ Int. Conf. Intelligent Robots and Systems, pp.215–221, IEEE, 2010.
[9] A.L. Rankin, L.H. Matthies, and P. Bellutta, “Daytime water detection based on sky reflections,” 2011 IEEE Int. Conf. Robot. Autom.,
pp.5329–5336, IEEE, 2011.
[10] M. Bertozzi, R.I. Fedriga, and C. D’Ambrosio, “Adverse driving
conditions alert: investigations on the swir bandwidth for road statusmonitoring,” Int. Conf. Image Analysis and Processing, pp.592–601,
Springer, 2013.
[11] A. Fisher, N. Flood, and T. Danaher, “Comparing landsat water index methods for automated water classification in eastern australia,”
Remote Sensing of Environment, vol.175, pp.167–182, March 2016.
[12] M. Shimano, H. Okawa, Y. Asano, R. Bise, K. Nishino, and I. Sato,“Wetness and color from a single multispectral image,” Proc. IEEE
Conf. Comput. Vis. Pattern Recognit., pp.3967–3975, 2017.
[13] H. Okawa, M. Shimano, Y. Asano, R. Bise, K. Nishino, and I. Sato,“Estimation of wetness and color from a single multispectral image,”
IEEE Trans. Pattern Anal. Mach. Intell., 2019.
[14] Y. Asano, Y. Zheng, K. Nishino, and I. Sato, “Shape from water: Bispectral light absorption for depth recovery,” European Conf.
Comput. Vis., pp.635–649, Springer, 2016.
[15] J.M. Bioucas-Dias, A. Plaza, G. Camps-Valls, P. Scheunders, N.
Nasrabadi, and J. Chanussot, “Hyperspectral remote sensing data
analysis and future challenges,” IEEE Geoscience and remote sensing magazine, vol.1, no.2, pp.6–36, June 2013.
[16] C.I. Chang, Q. Du, T.L. Sun, and M.L. Althouse, “A joint bandprioritization and band-decorrelation approach to band selection for
hyperspectral image classification,” IEEE Trans. geoscience and remote sensing, vol.37, no.6, pp.2631–2641, Nov. 1999.
[17] H. Ren and C.I. Chang, “Automatic spectral target recognition in
hyperspectral imagery,” IEEE Trans. Aerospace and Electronic Systems, vol.39, no.4, pp.1232–1249, Oct. 2003.
[18] A. Chakrabarti and T. Zickler, “Statistics of real-world hyperspectral
images,” CVPR 2011, pp.193–200, IEEE, 2011.
[19] J.P.S. Parkkinen, J. Hallikainen, and T. Jaaskelainen, “Characteristic
spectra of munsell colors,” JOSA A, vol.6, no.2, pp.318–322, 1989.
[20] T. Fawcett, “An introduction to roc analysis,” Pattern Recognit. Lett.,vol.27, no.8, pp.861–874, June 2006.
[21] Y. Monno, H. Teranaka, K. Yoshizaki, M. Tanaka, and M. Okutomi,“Single-sensor rgb-nir imaging: High-quality system design andprototype implementation,” IEEE Sensors Journal, vol.19, no.2,
pp.497–507, 2018.
[22] T.F. Chan and L.A. Vese, “Active contours without edges,” IEEE
Trans. Image Process., vol.10, no.2, pp.266–277, Feb. 2001.
[23] S.K. McFeeters, “The use of the normalized diﬀerence water index(ndwi) in the delineation of open water features,” International journal of remote sensing, vol.17, no.7, pp.1425–1432, 1996.
[24] H. Xie, X. Luo, X. Xu, X. Tong, Y. Jin, H. Pan, and B. Zhou, “Newhyperspectral diﬀerence water index for the extraction of urban water bodies by the use of airborne hyperspectral images,” J. Applied
Remote Sensing, vol.8, no.1, p.085098, 2014.
[25] I.S. Reed and X. Yu, “Adaptive multiple-band cfar detection of an
optical pattern with unknown spectral distribution,” IEEE Trans.
Acoust., Speech, Signal Process., vol.38, no.10, pp.1760–1770, Oct.

1990.
[26] T. Jaaskelainen, J. Parkkinen, and S. Toyooka, “Vector-subspacemodel for color representation,” JOSA A, vol.7, no.4, pp.725–730,
1990.

Chao Wang
received his B.S. in Computer
Science and Technology from Shenyang University of Technology in 2015. Also, he received
an M.S. in Advanced Information Engineeringfrom Kyushu Institute of Technology in 2018.
Currently, he is studying for a Ph.D. in Information Engineering from Kyushu Institute of Technology. His research includes computer visionand machine learning. He holds keen interestsin the area of hyperspectral imaging, and material classification.

Michihiko Okuyama
received his B.S.
in Intelligent Information Engineering from
Kyushu Institute of Technology in 2019. He
holds keen interests in the area of computer vision, and image processing.

Ryo Matsuoka
received the B.E., M.E.,
and Ph.D. degrees from The University of
Kitakyushu, Kitakyushu, Japan, in 2011, 2013,
and 2016, respectively. He has been an AssistantProfessor in the Faculty of Engineering, Kagawa
University from 2016 to 2018. In 2019, he has
also been an Assistant Professor in the Department of Artificial Intelligence, Kyushu Instituteof Technology. He is currently with the Facultyof Environmental Engineering, The Universityof Kitakyushu, as an Associate Professor. His
research interests are in image processing and computer vision. He received the Excellent Award of the Hibikino Award for master’s thesis andthe Excellent Student Award of the IEEE Fukuoka Section in 2013.

Takahiro Okabe
Takahiro Okabe received
his B.S. and M.S. degrees in physics, and his
Ph.D. degree in information science and technology from the University of Tokyo, Japan, in
1997, 1999, and 2011 respectively. After working at Institute of Industrial Science, the University of Tokyo, he joined Kyushu Institute ofTechnology, Japan, as an associate professor in
2013. He is currently with Department of Artificial Intelligence, Kyushu Institute of Technology, as a professor. He was a visiting scholar
at Tübingen University, Germany, from 2011 to 2012. His research interests include computer vision, computational photography, image processing, pattern recognition, and computer graphics, in particular their physicaland mathematical aspects. He received the IEICE Best Paper Awards in
2006, 2008, and 2010.