Noname manuscript No.
(will be inserted by the editor)

Image editing for weathering eﬀects with geometric details
Yuki Endo · Yoshihiro Kanamori · Jun Mitani · Yukio Fukui

Abstract We present an image-editing technique thatcan synthesize weathering eﬀects with ﬁne-scale geometric variations that occur together with weathering
phenomena. We extract such ﬁne-scale geometries as
high-frequency components of the image semi-automatically according to the estimated weathering distribution in the image. The geometric details are modiﬁed
when (de)-weathering is applied, yielding much more
realistic appearance than previous methods without adding excessive user inputs. We demonstrate the eﬀectiveness of the present technique through comparisonswith previous methods.
Keywords weathering and aging · image processing

1 Introduction
Outdoor objects in the real world change their appearance due to weathering over time. For example, metals
get rust, paints peel, and stones become mossy, erode,and so on. Reproduction of such weathering phenomena is very important to enhance the reality of object
appearance in computer graphics.
Several techniques have been proposed for modeling weathering eﬀects. These techniques can be broadlygrouped into sample-based approaches that handle general targets, and simulation-based methods that handle
speciﬁc targets. Here we brieﬂy introduce some of them,and refer to the survey paper by Merillou et al. [10]and the book by Dorsey et al. [7] for more information.
Physically-based simulations of weathering and aging
have been applied to various targets such as stones [6],paint peeling [11], rust [5] and moss [4]. These methods
are meant to handle 3D models and diﬃcult to apply
Y. Endo · Y. Kanamori · J. Mitani · Y. Fukui
Department of Computer Science, University of Tsukuba,
Ibaraki, Japan
E-mail: endo@npal.cs.tsukuba.ac.jp
J. Mitani
JST ERATO, Tokyo, Japanto a single image. In sample-based approaches, Gu etal. [8] and Lu et al. [9] reproduce temporal variationsof materials for 3D models by capturing real-world materials. However, these techniques cannot be applied toa single image because of the requirement of capturesof full time sequences.
Wang et al. [13] measured BRDFs at various points
on a single material that exhibits gradation of weathering degrees, and constructed an appearance manifoldfrom the BRDFs, which represents the temporal variation of the material. While this method mainly targets
at 3D models, Xue et al. [14] as well as Bandeira andWalter [2] proposed methods for reproducing weathering in 2D images. Both of their methods construct
the models of the time-variant appearance of objects,
appearance manifolds [14] or appearance maps [2], andcalculate the distribution of weathering degrees in theimage based on the models. The weathering degrees arethen used to decompose the image into reﬂectance andilluminance, which is often interpreted as shading in thiscontext. Consequently, (de)-weathering is accomplishedby changing only the reﬂectance according to weathering degrees while the shading remains unchanged. This
framework allows applications to objects with complicated shading, but cannot handle time-varying shading due to weathering, that is, shading caused by timevarying geometric details of rust, peeled paint or moss.
Recently, Bandeira and Walter [3] tackled this problem by using normal mapping. Their approach seems to
successfully reproduce the examples of cracks. However,the propagation of cracks is separately handled without
using weathering degrees, and thus their method cannot handle the cases where both shading and reﬂectancechange simultaneously according to (de)-weathering.
In this paper, we propose a technique for modeling weathering eﬀects with time-varying geometric details in images. Speciﬁcally, we focus on spatio-temporalvariations of shading as well as reﬂectance due to weathering. Because such variations of shading often appear
in high-frequency components of the shading image, weon the least weathered point and the most weathered
point selected by the user. Weathering degree values
are within the range of [0, 1] where 0 indicates the least
weathered and 1 indicates the most weathered.
The reﬂectance luminance values are the average ofluminance values of pixels whose weathering degrees
are almost the same. The shading luminance values arecalculated as S(i, j) = Il (i, j)/Wl (i, j). After the decomposition, the weathering degree map is updated,
and then the deweathered or weathered reﬂectance iscomputed with the appearance map and the weathering degree map. Finally, a resulting image is obtainedby combining reﬂectance with shading.

Fig. 1 Example images of rust eﬀects. Top left: the input
image. Bottom left: a resultant image by Bandeira and Walter [2]. Bottom right: a resultant image by our method, withgeometric details caused by rusting. Each magniﬁed ﬁgure isshown at the top right.

3 Modeling weathering eﬀects with geometricdetails

We handle (de)-weathering eﬀects where both shading
and reﬂectance change simultaneously. Following themethod of Bandeira and Walter [2], the region of interest (ROI) in the input image is decomposed into shadextract the high-frequency components and add theming and reﬂectance based on a weathering degree map
onto object surfaces when weathering progresses to acomputed with the appearance map.
certain extent. This extraction can be done with only a
The left photograph in Fig. 2 exhibits a typical exfew additional user inputs. (De)-weathering with timeample where not only reﬂectance but also shading varies
varying shading is accomplished in our work for thedue to weathering. In this case, rusting causes ﬁne-scaleﬁrst time, and substantially enhances the reality of (de)geometric variations on the iron surface. As illustrated
weathering eﬀects as shown in Fig. 1. Because our method in the right graph in Fig. 2, the variations are observed
is based on Bandeira and Walter’s algorithm using apas a high-frequency pattern in the image, which is simpearance maps [2], we provide a brief overview of theirilar in other weathering examples such as peeling paintalgorithm in the next section.
and moss as demonstrated in this paper.
Based on this observation and an assumption thatthe texture pattern on the object surface can be ig2 The method of Bandeira and Walter
nored, we consider the high-frequency components inshading as the geometric variations caused by weatherBandeira and Walter’s method [2] models an input iming. As a preprocessing, we extract the high-frequency
age as the product of reﬂectance and shading, and aspatterns and ﬁll the ROI with the patterns using a texsumes that only the reﬂectance changes due to weathture synthesis method. Then, we add these synthesizedering phenomena.
patterns onto shading according to the updated weathTo decompose an input image into reﬂectance andering degrees. Reﬂectance is also calculated similarly toshading, they use Lab color space that allows to hanBandeira and Walter’s method [2]. Finally, we obtaindle luminance (L) and chroma (ab) separately. They
resulting images by multiplying reﬂectance and shadassume that chroma values of shading are constant anding. In the following sections, we describe each of thesechroma values of reﬂectance are given as ab channels
steps in detail. Although our approach cannot be apof each pixel. On the other hand, luminance values ofplied to images with high-frequency texture patterns
reﬂectance and shading are modeled as follows.
and/or many colors, it reproduces realistic results compared to previous approaches in many applications.
Il (i, j) = Wl (i, j) × S(i, j)(1)where i and j are horizontal and vertical coordinatesof each pixel, Il is luminance of the image, Wl is reﬂectance luminance represented as the weathering component, and S is shading luminance. The reﬂectance luminance Wl are computed based on a weathering degreemap.
The weathering degree map is calculated with an
appearance map that is constructed in advance based

3.1 Extraction of shading details
Given a shading image S, we decompose S into coarse
features and ﬁne features using a multi-scale image decomposition presented by Subr et al. [12]. Because thisdecomposition can separate the ﬁne texture from shading if the texture and shading are of diﬀerent scales,

Intensity

Weathered region

PixelsFig. 2 A typical example of the fact that weathered regions often become rough. Left: A photograph of a rusting surface.
Middle: shading of the left image. Right: intensity plots of shading along the red scanline.

Weathering
degree map(default) d

Shading S

Weathering
degree map(weathered) d’

Reflectance(weathered) W’

Coarse features SC

Fine features Sf

Resulting image

Fig. 3 Decomposition into coarse features and ﬁne features.
Synthesizedfine features S’f

Shading(default) S

Shading(weathered) S’

The region selected by the user

Fig. 5 An overview of synthesis of weathering eﬀects with geometric variations. The weathered shading is calculated basedon the weathering degree map and the synthesized ﬁne features. The resultant image is obtained by multiplying theprecomputed reﬂectance and synthesized shading.

Weathering
degree map
Texture(default) dSynthesis

Fine features of shading Sf

Synthesized fine features S’f

Fig. 4 An outline of the synthesis. Fine features of shading
are synthesized in a weathered region based on a weathering
degree map.

it works well for our purpose of extracting the highfrequency patterns. The shading image S is decomposed
into coarse features Sc and ﬁne features Sf as follows:
S(i, j) = Sc (i, j) + Sf (i, j)
The decomposed result is shown in Fig. 3.

3.2 Texture synthesis(2)

where t denotes a weathering threshold. We found t =0.5 works well for the results in this paper. The source
texture pattern is sampled from regions where M (i, j) =1 in order to obtain synthesized ﬁne features Sf′ .
Note that the texture synthesis may cause artifacts
if the binary mask contains shadowed or non-weatheredregions. This problem can be avoided by letting theuser to manually specify a rectangular region in which
the binary mask is constructed and the source texture
is sampled. Fortunately, such additional user input is
not required in most cases by setting the rectangularregion around the most weathered point speciﬁed whenconstructing the appearance map.
3.3 Weathering with geometric details

To ﬁll the ROI with extracted high-frequency patterns,
After the preprocessing described above, the resultant
we employ a texture synthesis algorithm by Ashikhmin [1]. shading is computed with the synthesized ﬁne shadingThis algorithm runs very fast and constructs resultscomponents Sf′ , the initial shading S, the initial weaththat often look good. Although it sometimes brings
ering degree map d and the updated weathering degree
abrupt discontinuity depending on the input texture,map d′ as shown in Fig. 5. The ﬁne components are
it suﬃces for our purpose because the input texture in
added to the regions where weathering progresses to a
our case is just a rough pattern that does not require
certain extent:{
continuity.
S(i, j) + Sf′ (i, j) (d′ (i, j) − d(i, j) ≥ t)
′
(4)
S
(i,
j)
=
The synthesis process is shown in Fig. 4. To ensure
S(i, j)(d′ (i, j) − d(i, j) < t)that the source texture pattern is obtained only fromOn the other hand, the reﬂectance variations are calcuthe weathered region, we calculate a binary mask M
lated using the appearance map according to the weathfrom the weathering distribution map d.
ering degree for each pixel. Finally, a resultant image{
with detailed shading is obtained by multiplying the1 (d(i, j) ≥ t)
M (i, j) =(3)shading and reﬂectance values.
0 (d(i, j) < t)

Fig. 6 A comparison of de-weathering processes. Left toright: the input image, a de-weathered result by Xue etal. [14], a de-weathered result by Bandeira and Walter [2]and a de-weathered result by our method. Magniﬁed images
are shown at upper left of the results. The original smoothsurface is well reconstructed by our method while the geometric details due to rust remain in the results of the previous
methods.

We also try de-weathering taking into account thegeometric variations, i.e., recovery of the original surfaces of objects before going through weathering. Thisis a challenging task in general because the original surfaces are unknown. In contrast to weathering describedabove, we synthesize the high-frequency components ofshading extracted from non-weathered regions. A comparison of de-weathering eﬀects between the previous
methods [14, 2] and our method is shown in Fig. 6.
The original smooth surface is well reconstructed byour method while the geometric details due to rust remain in the results of the previous methods.

4 Results
Our implementation was written in C++, using OpenGL,
GLUT and GLUI. We ran our program on a PC with anIntel Core i7 2.80 GHz CPU and an NVIDIA Quadro
FX 580 GPU. The input images we used in Fig. 2 andFig. 6 were directly taken from the paper of Xue etal. [14], and the others were downloaded from ﬂickr(http://www.ﬂickr.com/).
The sizes of the input images and the time for preprocessing (Sections 3.1 and 3.2) are within 265 × 333
pixels and 5 seconds. The time required to update aweathering degree map and to calculate reﬂectance andshading is almost the same as Bandeira and Walter’s
method [2] because the additional process is just to incorporate the high-frequency components that are synthesized in the preprocessing.
Fig. 1 and Fig. 7 show comparisons between our
method and the previous method [2] to reproduce rusting and mossy eﬀects. Note that in our results, theweathering eﬀects with geometric details look more realistic compared to the results by the previous method.

5 Conclusion and future work
We have presented a technique for reproducing weathering eﬀects taking into account the geometric detailscaused by weathering. Because we simply assume that

Fig. 7 Comparisons of mossy eﬀects. Left: the input image. Middle: the resultant image by Bandeira and Walter [2].
Right: the resultant image by our method. Moss propagateswith geometric details in our method while only the color
changes in the previous method.

geometric variations due to weathering appear as highfrequency patterns, we cannot handle intrinsic highfrequency texture patterns and low-frequency weathering eﬀects. Nevertheless, we have demonstrated that
our approach can achieve more realistic results compared to previous methods. For future work, we would
like to develop a better deweathering technique thatcan restore the shapes of objects.
References
1. Ashikhmin, M.: Synthesizing Natural Textures. 2001 ACM
Symposium on Interactive 3D Graphics, pp.217-226, 2001.
2. Bandeira, D., Walter, M.: Synthesis and Transfer of TimeVariant Material Appearance on Images, Sibgrapi, pp.3239, 2009.
3. Bandeira, D., Walter, M.: Highlights on weathering eﬀects:
Improving the appearance modeling of weathering eﬀects
on images, The Visual Computer, 26, 6-8, 2010.
4. Desbenoit, B., Galin, E., Akkouche, S.: Simulating andModeling Lichen Growth, Computer Grphics Forum, 23, 3,
pp.341-350, 2004.
5. Dorsey, J., Hanrahan, P.: Modeling and Rendering ofMetallic Patinas, In Proc. of SIGGRAPH, pp.387-396, 1996.
6. Dorsey, J., Edelman, A., Jensen, W. H, Legakis, J., Pedersen, H.: Modeling and Rendering of Weathered Stone. In
Proc. of SIGGRAPH, pp.225-234, 1999.
7. Dorsey, J., Rushmeier, H., Sillion, F.: Digital Modelingof Material Appearance. Morgan Kaufmann/Elsevier, San
Mateo/ Amsterdam, 2007.
8. Gu, J., Tu, C., Ramamoorthi, R., Belhumeur, P., Matusik,
W., Nayar, S.: Time-Varying Surface Appearance: acquisition, modeling and rendering, ACM Transactions on Graphics, 25, 3, pp.762-771, 2006.
9. Lu, J., Georghiades, A. S., Glaser, A., Wu, H., Wei, L.,
Guo, B., Dorsey, J., Rushmeier, H.: Context-Aware Textures, ACM Transactions on Graphics, 26, 1, 2007.
10. Merillou, S., Ghazanfarpour, D.: A survey of aging andweathering phenomena in computer graphics, Computers &Graphics 32, pp.159-174, 2008.
11. Paquette, E., Poulin, P., Drettakis, D.: The Simulation of
Paint Cracking and Peeling, In Proc. of Graphics Interface, pp.59-68, 2002.
12. Subr, K., Soler, C., Durand, F.: Edge-preserving Multiscale Image Decompostion based on Local Extrema, ACM
Transactions on Graphics, 28, 5, 2009.
13. Wang, J., Tong, X., Lin, S., Pan, M., Wang, C., Bao,
H., Guo, B., Shum, H.: Appearance manifolds for modelingtime-variant appearance of materials, ACM Transactions
on Graphics, 25, 3, pp.754-761, 2006.
14. Xue, S., Wang, J., Tong, X., Dai, Q., Guo, B.: Imagebased Material Weathering, Computer Grphics Forum, 27,2, pp.617-626, 2008.