Realistic Modeling of Water Droplets for Monocular
Adherent Raindrop Recognition using Bézier Curves
Martin Roser, Julian Kurz and Andreas Geiger
Department of Measurement and Control
Karlsruhe Institute of Technology (KIT)
D-76131 Karlsruhe, Germany

Abstract. In this paper, we propose a novel raindrop shape model for the detection of view-disturbing, adherent raindrops on inclined surfaces. Whereas stateof-the-art techniques do not consider inclined surfaces because they assume thedroplets as sphere sections with equal contact angles, our model incorporates cubic Bézier curves that provide a low dimensional and physically interpretable representation of a raindrop surface. The parameters are empirically deduced fromnumerous observations of different raindrop sizes and surface inclination angles.
It can be easily integrated into a probabilistic framework for raindrop recognition,using geometrical optics to simulate the visual raindrop appearance. In comparison to a sphere section model, the proposed model yields an improved droplet
surface accuracy up to three orders of magnitude.

1 Introduction
Outdoor navigation and surveillance demand for reliable and robust computer visionalgorithms. They have to meet stringent conditions concerning disturbances caused byarbitrary weather conditions. In fact, there are various atmospheric influences which restrict the usability of these systems such as fog, rain or snow. Especially in rainy weather
it is often the case that adherent waterdrops on the lens-protecting glass disturb the viewof a camera. Although a lot of research has been pursued in robotics [16], computer
vision [5, 7, 11] and for driver assistance [9, 12, 19], raindrop detection still remains achallenging task. This might be for several reasons: Water droplets on a glass surface
exhibit a large variety in shape and size. Transparency makes their appearance highly
dependent on the image background. Moreover, water droplets on the protecting glassof a camera are subject to severe out-of-focus blur which lowers their distinguishability
from the scene background.
Recent work on raindrop detection [8, 12] assumes a simple sphere section for modeling the droplet boundary. Especially on tilted planes where gravity causes an unidirectional droplet deformation, this assumption does not hold. While high-order polynomials are more adequate, a physical interpretation of the fitted parameters is hard.
In this paper we propose a novel raindrop shape model, that provides a physicallyinterpretable parameter set of low dimensionality. Our main contribution is a model
based on cubic Bézier curves. We provide a broad validation of the model parametersand show, that the shape deviation between fitted model and real droplet will be significantly decreased compared to state-of-the-art sphere section models. The proposed
shape model can be easily integrated into existing raindrop recognition frameworks.

2

M. Roser, J. Kurz and A. Geiger

2 Related WorkThe visual effects of rain are manifold and complex. Water droplets in the atmospherelead to contrast attenuation in the far-field of the camera, whereas falling raindrops
produce sharp intensity changes in image sequences. Adherent raindrops in front of
the camera lens disturb the view from the camera and light reflections on the droplet
surfaces additionally deteriorate computer vision algorithms.
Related work on dynamic weather effects like the appearance of falling raindrops
in image sequences has been performed by [5, 7]. They studied the influence of fallingraindrops on the image acquisition process and introduced a photometric model for
spherical raindrops in the atmosphere that is used for enhanced video processing likeremoving rain from image sequences [4] or rain streak rendering [6].
In the targeted context of outdoor navigation and surveillance, falling raindrops andrain streaks can be considered as atmospheric noise and are not the dominant disturbing
effect. Stronger limitations are imposed by adherent water droplets on the glass surface
covering the camera lens. Kurihata et al. [9] used a machine learning approach withraindrop templates to detect raindrops on windshields from inside a moving vehicle.
Results within the sky area were promising, whereas the proposed method produceda large number of false positives within the non-sky regions of the image where raindrop appearance modeling becomes more challenging. In this work, in contrast, we
aim to accurately exploit the physical relationship between droplets shape and theirappearance. Zhang et al. [19] combined a wavelet transform for image blur detection with motion analysis using cumulative differences to recognize optical contaminations close to the camera. Their approach works well for rigid, opaque contaminations but fails in the presence of raindrops because their appearance strongly depends
on the scene background. Yamashita et al. exploited hardware constraints like multiple
cameras [13, 18] or pan-tilt surveillance cameras [16, 17] with known yaw rates in order
to bypass the challenge of modeling the complex optical behavior of raindrops. Roseret al. [8, 12] simulated spherical droplets on a glass surface using geometrical optics
and out-of-focus blur for the task of raindrop detection. However, they lack a realistic
shape parametrization of droplets that in practice are subject to gravity.
The remainder of this paper is structured as follows: Section 3 discusses the propagation of light rays through a droplet and shows how it can be used for raindrop detection. In Section 4 we proposes a raindrop shape model based on a Bézier curve representation. Validation results on real data and a comparison to a sphere section model [8]are given in Section 5.

3 Raindrop Recognition
Given a set of n artificial raindrop hypotheses D1 , . . . , Dn for different image positions
and presumed drop radii, the raindrop recognition task can be formulated as computing
the MAP estimate of the conditional probabilityp(d|z) ∝ p(d)p(z|d)(1)

with respect to the pattern d ∈ {⊘, D1 , . . . , Dn }. Here ⊘ is a background pattern thatmodels the case where the image region is not disturbed by a raindrop and z(u, v, r)

Realistic Modeling of Water Droplets3

Protecting
glass

Scene

P2
x
x

P1 Camera

D

Image planeOccludedbackground

Adherent
raindrop

Transitions(a) 3D raindrop refraction model

Dblurred(b) Artificial raindrop patternFig. 1. Droplet refraction model. (a) depicts the image formation process in the presence ofraindrops on a protecting glass in front of the camera, using geometrical optics. In (b) an artificial
raindrop pattern D is rendered by tracing the light rays through the raindrop to the backgroundand composing all found background pixels. For demonstration purposes, an additional out-offocus blur is applied and the blurred pattern Dblurred is added to the original image.

denotes the observation at position u, v with radius r in form of local image statistics.
It can be achieved densely as wells as from preselected points of interest, like CenSurE[1] or SURF [2]. Fairly standard cost measures such as the Sum-of-Absolute Differences (SAD) or the Sum-of-Squared Differences (SSD) are applied for modeling theobservation likelihood p(z|d). The prior may model the occurrence probability for different raindrop sizes in various adverse weather conditions in an empirical Bayesian
perspective according to [15].
Raindrop hypotheses for any circular region x = (u, v, r)T are achieved from observed points in the environment, using geometrical optics. As depicted in Fig. 1(a), alight ray emanating from a point in the environment will be refracted by the raindrop
and the protecting glass surface multiple times and reaches the camera sensor at point
P1 . Unless the raindrop does not occlude this environment point, it will be sensed asecond time at point P2 . Note, that the droplet acts as a convex lens with a small focal
length. Hence, for typical application in navigation and surveillance it is ensured that
only a minor part of the environment points are occluded by the raindrop (see Fig. 1(a)).
An accurate geometric relationship between P1 and P2 can be derived using Snell’s
law of refraction as shown in Fig. 1(b). Note, that the refraction on the protecting glass
occurs with respect to the (constant) plane normal np of the protecting glass, whereas ageneral drop surface S exhibits a normal field NS that can be deduced from the chosen
drop parametrization. Whereas [8, 12] use simple sphere sections that are in general
3D surfaces of constant curvature as depicted in Fig. 2(a), this model approximates theraindrop shape only insufficiently and results in a high model deviation especially when
dealing with tilted glass surfaces. For this reason, we derive a raindrop shape model regarding numerous tilt angles and drop sizes by using two orthogonal oriented Bézier
curves as illustrated in Fig. 2(b).

4

M. Roser, J. Kurz and A. Geiger
z

nS

x
y

ny

z
y

nx z
x

(a) Sphere section model(b) Proposed shape modelFig. 2. Raindrop surface models. (a) shows the droplet surface and its surface normals for asphere section model. In (b) a 3D model is created by superposing two orthogonal Bézier curves.

4 Raindrop Shape Model
4.1

Bézier Representation

Droplets on a horizontally aligned surface are symmetrical and have equal contact
angles. When neglecting any gravity, they can be characterized adequately, using thesphere section model as described in [8, 12]. However, gravity leads to a flattened raindrop surface shape which results in an inaccurate droplet modelling when assuming
sphere sections. On tilted surfaces the sphere section model assumption is violated evenmore, because the unsymmetrically applied gravity force will shift the droplet centroid
towards the declining direction, which yields different contact angles and a distinctly
bellied shape as illustrated in Fig. 5.
The shape of a raindrop can be described by parametric functions, like polynomialsof arbitrary order, Taylor polynomials or Bézier curves [3]. Here we employ Bézier
curves, since they describe real water droplets accurately and they provide an intuitive,
low-dimensional parameter set with a credible physical interpretation. This makes theverification of the model and an approximation for different angles and drop volumes
more transparent than interpreting the coefficients of a polynomial fit.
A Bézier curve of the degree n is characterized by a control polygon consisting ofnn + 1 Bézier points (Pi )i=0 , P ∈ R2 . It is defined in an interval t ∈ [0 . . . 1] as
C(t) =nX

Bi,n (t)Pi ,

(2)

 
n i
t (1 − t)n−i
i

(3)

i=0

whereasBi,n (t) =indicates the Bernstein polynomial i of degree n [3].
A cubic Bézier curve (n = 3) has sufficient degrees of freedom to describe the raindrop shape well. As depicted in Fig. 3, a capable interpretation of the Bézier points
3(Pi )i=0 can be achieved by transforming them to the contact angles α1 , α2 of thedroplet that are originated from physics of boundaries and the weight factors w1 , w2

Realistic Modeling of Water Droplets

5

mean Bézier curve
control polygon
Bézier points

Fig. 3. Cubic Bézier curve representation. The Bézier points are transformed physically interpretable: α1 , α2 represent the droplets contact angles and the weight factors w1 , w2 are related
to the influence of gravity for inclined surfaces.

(a)

(b)

(c)

(d)

Fig. 4. Image processing for drop shape extraction. (a) shows the original image taken in theexperimental setup. A distinction between surface plane and raindrop points is performed by
RANSAC line fitting in the Canny image (b). In order to remove further outliers, two second
order polynomials are fitted robustly to the left (red) and right (green) side of the raindrop (c).
Finally, least squares Bézier curve fitting is performed on all inlier points (d).

that are related to the centroid shift due to gravity.
α1 =∠(P0 P1 , P0 P3 )(4)

α2 =∠(P2 P3 , P0 P3 )(5)

w1 =P0 P1(6)

w2 =P2 P3(7)

Finally, the curvature normals of two orthogonal, cubic Bézier curves form a 3D
droplet surface S as illustrated in Fig. 2(b)   0nx1
 0  + ny  ,
(8)
nS =
T
T
|| (nx , 0, 1) + (0, ny , 1) ||1
1where one curvature represents the side view with the inclination angle of the lensprotecting glass and the other representing the top view with θ = 0◦ .
4.2

Bézier curve fitting

In order to characterize and describe the drop shape in terms of cubic Bézier curves,
we performed an image pre-processing as described briefly in the following paragraph.

6

M. Roser, J. Kurz and A. Geiger

An overview of the image processing and curve fitting methods can be found in Fig. 4.
The first step of extracting the drop shape is to take raw observations from the cannyedge image (Fig. 4(b)). A robust RANSAC line fitting approach estimates the remaining glass surface direction and hence compensates small errors due to inaccuracies in
the angular arrangement of glass plate and camera. In order to further remove outliers
from the measurements, two parabolas were fitted through the remaining points, using RANSAC: one from the maximum to the left side (red line in Fig. 4(c)) and one
to the right side (green line in Fig. 4(c)). Note, that we do not use the parameters of
the parabola fits directly because the shape is neither described consistently nor interpretable in a physical way. Instead, a combination of all inliers gives a set of points thatis used for the subsequent Bézier curve fitting as shown in Fig. 4(d). The Bézier curve
fitting is performed in a least squares sense [14] by splitting (2) into two independentequations for the x and y coordinates
x = ax t3 + bx t2 + cx t + dx
3

(9)

2

y = ay t + by t + cy t + dy ,

(10)

3

and computing the Bézier points (Pi )i=0 by comparing coefficients to (2). The factort ∈ [0 . . . 1] corresponds to the normalized curvature length. For a curve described by
N points t is approximated byn pP∆x(k)2 + ∆y(k)2t(n) = k=1
N p
P

,

(11)

∆x(l)2 + ∆y(l)2l=1

where ∆x and ∆y are the differences between two neighboring points.
Repeating the experiments M times for each drop volume and inclination angle
configuration, we receive M different Bézier curve parameterizations. A mean Bézier
3
curve is finally achieved by computing the mean of each Bézier point (Pi )i=0 :
Pi =

M
X
Pi k

k=1

M

.

(12)

5 Results
For all experiments, a digital camera was mounted next to a tiltable glass plate to capture
the shape of water droplets of different sizes under multiple inclination angles. We used
an Eppendorf Research Plus pipette for all experiments in order to guarantee a precise
but adjustable drop volume size. In the experimental setup all drops are illuminated bya lamp in front of a dark background to achieve a good contrast and ensure reliable
shape extraction. As input for finding an empirical description of water droplets on aflat surface, multiple images with different drop sizes and surface inclination angles
were taken. The chosen drop volumes for our test series were 5, 10, 15 and 20µl andthe inclination angles of the glass plate were 0◦ , 25◦ , 30◦ , 35◦ and 40◦ . The chosenRealistic Modeling of Water Droplets
inclination
angle

volumevolumevolume7

volume

0°

25°

30°

35°

40°

Fig. 5. Experiments. Sample imagery for manifold drop volumes and surface inclination angles.
For model estimation the mean fit of 20 images for each drop volume and inclination angle setting
is used.

drop volumes in µl correspond to 1.06, 1.34, 1.53 and 1.7 mm drop radii of fallingraindrops, which was motivated by [7] who proposes probable raindrop radii between
0.5 − 2.5 mm. The experiments were repeated 20 times for each drop volume andinclination angle configuration. Hence, 400 raindrop shape images were acquired in
total. An overview of the different setup properties and their effects on the droplet shapeis depicted in Fig. 5.
The results section is divided into two parts: First, we discuss the estimated raindrop parameters. Then we present a comparison of the proposed model with the spheresection model of [8].
5.1Model parameters

A model capable of generating realistic droplet surfaces demands for a low dimensionalparametrization to avoid overfitting. In this section, we discuss the obtained dependencies of the Bézier curve based model with respect to the design parameters (drop volume
and inclination angle).
Assuming the raindrop diameter d = |P0 P3 |, the upper row in Fig. 6 shows the expected behavior that the drop radius increases with its volume. A tendency of increasing
drop diameters for larger inclination angles exists, although it may not be the predominant effect. This phenomenon can be explained from the droplet area that looses its
circular shape and develops a predominant direction with increasing inclination angles.
Hence, even if the drop volume is not a-priori known like it is the case in image-basedraindrop detection tasks, for a given surface inclination angle the volume can be estimated from the observed drop diameter. In principle, the scale of standard multi-scaleinterest point detectors like SURF [2] would provide sufficient information for that task.

8

M. Roser, J. Kurz and A. Geiger
volumevolumevolumevolumeBézier weights

contact angles

diameter

d
d

0

10

20

30

40 0

10

20

30

40 0

10

20

30

40 0

10

20

30

40

Fig. 6. Model parameter. The first row shows the averaged droplet diameter d as a function of
the inclination angle θ for different raindrop volumes (columns). The second and the third row
depict the mean contact angles α1 , α2 and the mean Bézier weights w1 , w2 , respectively.

Tilting the glass surface leads to a deformation of the drop due to changed gravity
influences. For this reason, we expect an increasing difference ∆ = |α2 − α1 | between
both contact angles. The middle row in Fig. 6 shows the expected behavior, although not
all contact angles could be extracted accurately, throughout the experiments. However,α1 tends to decrease with increasing inclination angle. α2 shows a slight ascent but
decreases for θ = 40◦ . This can be explained by having a deeper look at the performedexperiments. We are only interested in stationary droplets. For θ ≈ 40◦ , the drop begins
rinsing down and hence we could not acquire representative imagery data.
The Bézier weight w1 remains constant for varying inclination angle and ascends
with increasing drop volume. For the right side w2 increases with inclination angle anddrop volume while for angles θ ≈ 40◦ the drop begins to move, again. As discussed
above, for these inclination angles, no reliable conclusion can be drawn.
In conclusion, a physically correct droplet shape can be derived, as soon as theinclination angle and the drop volume are given. The drop volume can be deduced fromthe observed raindrop diameter, whereas the surface inclination angle is given by thedefined camera mounting. This makes the proposed droplet shape model applicable for
an image-based raindrop detection approach.

Realistic Modeling of Water Droplets
volumevolumevolume9volume

SSD error

Bézier curve
Circle05

10 15 20 25 30 35 40 0

inclination angle

5

10 15 20 25 30 35 40 0

inclination angle

5

10 15 20 25 30 35 40 0

inclination angle

5

10 15 20 25 30 35 40

inclination angle

Fig. 7. Model accuracy. (a)-(d) show the SSD error of a sphere section model and the proposedmodel, using cubic Bézier curves.

5.2

Comparison

For comparing the accuracy of the proposed method to state-of-the art, a 2D cut of asphere section was fitted to the extracted raindrop surface measurements using nonlinear Levenberg-Marquardt optimization [10]. The error measure is defined in terms of
Sum-of-Squared Differences (SSD).
Fig. 7 shows the error generated using the sphere section model in comparison to the
new Bézier curve based model. Even for flat surfaces (θ ≈ 0◦ ) and small drop volumes,
the proposed model has an SSD error which is three order of magnitude smaller. Thisillustrates the importance to take into account the gravity force which flattens the drop
surface. An increasing drop volume and inclination angle lead to unsymmetrical droplet
deformation, which emphasizes the advantage of the proposed shape model with respect
to the sphere section model.

6 Conclusion
In this paper we proposed a novel raindrop shape model based on cubic Bézier curves
and showed its potential for its integration in image-based raindrop detection approaches.
The model was deduced from numerous experiments on water drops of various volumes on a flat surface with different inclination angles. A physically correct droplet
shape could be computed if just the inclination angle and the drop volume were given.
The drop volume was deduced from the observed raindrop diameter. This makes the
proposed droplet shape model applicable for an image-based raindrop detection approach. Finally, we showed that the shape deviation between the estimated Bézier curve
based model and the real droplet was significantly decreased compared to state-of-theart sphere section models.

7 Acknowledgment
The authors would like to thank the Karlsruhe School of Optics and Photonics (KSOP).

10

M. Roser, J. Kurz and A. Geiger

References
1. M. Agrawal, K. Konolige, and M.R. Blas. CenSurE: Center Surround Extremas for Realtime
Feature Detection and Matching., volume 5305 of Lecture Notes in Computer Science, pages
102–115. Springer, 2008.
2. H. Bay, T. Tuytelaars, and L. Van Gool. Surf: Speeded up robust features. In European
Conference on Computer Vision (ECCV ’06), Graz Austria, May 2006.
3. G. Farin. Curves and surfaces for CAGD: a practical guide. Morgan Kaufmann Publishers
Inc., San Francisco, CA, USA, 2002.
4. K. Garg and S.K. Nayar. Detection and removal of rain from videos. In IEEE Conference on
Computer Vision and Pattern Recognition (CVPR ’04), volume I, pages 528–535, Jun 2004.
5. K. Garg and S.K. Nayar. When does a camera see rain? In IEEE International Conference
on Computer Vision (ICCV ’05), volume 2, pages 1067–1074, Oct 2005.
6. K. Garg and S.K. Nayar. Photorealistic rendering of rain streaks. ACM Transactions on
Graphics, Jul 2006.
7. K. Garg and S.K. Nayar. Vision and rain. Internatl. Journal of Computer Vision, 75(1):3–27,
2007.
8. J. Halimeh and M. Roser. Raindrop detection on car windshields using geometricphotometric environment construction and intensity-based correlation. In IEEE Intelligent
Vehicle Symposium (IV ’09), Xi’an, China, 2009.
9. H. Kurihata, T. Takahashi, I. Ide, Y. Mekade, H. Muraseand Y. Tamatsu, and T. Miyahara.
Rainy weather recognition from in-vehicle camera images for driver assistance. In IEEE
Intelligent Vehicles Symposium (IV ’05), pages 205–210, 2005.
10. D. Marquardt. An algorithm for least-squares estimation of nonlinear parameters. SIAM
Journal of Applied Mathematics, 11:431–441, 1963.
11. S.G. Narasimhan and S.K. Nayar. Vision and the atmosphere. International Journal ofComputer Vision, 48(3):233–254, 2002.
12. M. Roser and A. Geiger. Video-based raindrop detection for improved image registration.
In IEEE Workshop on Video-Oriented Object and Event Classification (in conjunction withICCV ’09), 2009.
13. Y. Tanaka, A. Yamashita, T. Kaneko, and K.T. Miura. Removal of adherent waterdrops fromimages acquired with a stereo camera system. IEICE - Transactions on Information Systems,
E89-D(7):2021–2027, 2006.
14. P. J. G. Teunissen. Adjustment theory: an introduction. Delft University Press, Postbus 98,
2600 MG Delft, 2000.
15. Nuno Vasconcelos and Andrew Lippman. Empirical bayesian em-based motion segmentation. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Puerto Rico,
pages 527–532, 1997.
16. A. Yamashita, I. Fukuchi, T. Kaneko, and K.T. Miura. Removal of adherent noises fromimage sequences by spatio-temporal image processing. IEEE International Conference on
Robotics and Automation (ICRA ’08), pages 2386–2391, May 2008.
17. A. Yamashita, T. Harada, T. Kaneko, and K.T. Miura. Removal of adherent noises fromimages of dynamic scenes by using a pan-tilt camera. IEEE/RSJ International Conferenceon Intelligent Robots and Systems (IROS ’04), 1:437–442 vol.1, Sept.-2 Oct. 2004.
18. A. Yamashita, M. Kuramoto, T. Kaneko, and K.T. Miura. A virtual wiper - restoration ofdeteriorated images by using multiple cameras. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS ’03), 4:3126–3131 vol.3, Oct. 2003.
19. Y. Zhang, J. Yang, K. Liu, and X. Zhang. Self-detection of optical contamination or occlusion
in vehicle vision systems. Journal of Optical Engineering, 47(6):067006, 2008.