IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATE

1

Improving Color Constancy by Photometric
Edge Weighting
Arjan Gijsenij, Member, IEEE, Theo Gevers, Member, IEEE, Joost van de Weijer, Member, IEEE
Abstract—Edge-based color constancy methods make use of image derivatives to estimate the illuminant. However, different
edge types exist in real-world images such as material, shadow and highlight edges. These different edge types may have a
distinctive influence on the performance of the illuminant estimation.
Therefore, in this paper, an extensive analysis is provided of different edge types on the performance of edge-based color
constancy methods. First, an edge-based taxonomy is presented classifying edge types based on their photometric properties(e.g. material, shadow-geometry and highlights). Then, a performance evaluation of edge-based color constancy is provided
using these different edge types. From this performance evaluation it is derived that specular and shadow edge types are morevaluable than material edges for the estimation of the illuminant. To this end, the (iterative) weighted Grey-Edge algorithm is
proposed in which these edge types are more emphasized for the estimation of the illuminant.
Images that are recorded under controlled circumstances demonstrate that the proposed iterative weighted Grey-Edge algorithm
based on highlights reduces the median angular error with approximately 25%. In an uncontrolled environment, improvements inangular error up to 11% are obtained with respect to regular edge-based color constancy.
Index Terms—Color Constancy, Illuminant estimation, Grey-Edge, Edge classification

F

1

I NTRODUCTIONCHANGES in illumination cause the measurementsof object colors to be biased towards the colorof the light source. The goal of color constancy is
to provide invariance with respect to these changes.
Color constancy facilitates many computer vision related tasks like color feature extraction [1] and color
appearance models [2].
Many computational color constancy algorithms
have been proposed, see e.g. [3, 4] for an overview.
Traditionally, computational color constancy methods
use pixel values of an image to estimate the illuminant. Examples of such methods include approaches
based on low-level features, e.g. [5, 6, 7] and gamutbased algorithms [8, 9, 10]. Recently, methods that use
derivatives (i.e. edges) and even higher-order statistics
are proposed [11, 12, 13, 14].
The underlying assumption of Grey-World andGrey-Edge-based algorithms is that the distributionof the colors and edges is directed towards the illuminant direction. Hence, in order to accurately recover
the color of the light source, ideally, only those pixels
and edges should be used that coincide with the illuminant direction (highlights are one example of suchpixels, perfect reflectances are another). Under the• Arjan Gijsenij is with Alten PTS, the Netherlands.
• Theo Gevers is with the Intelligent Systems Lab Amsterdam (ISLA),
University of Amsterdam, The Netherlands and with the Computer
Vision Center (CVC), Campus Universitat Autónoma de Barcelona,
Spain.
• Joost van de Weijer is with the Computer Vision Center (CVC),
Campus Universitat Autónoma de Barcelona, Spain.
• e-mail: arjan.gijsenij@gmail.com; th.gevers@uva.nl; joost@cvc.uab.es

assumption of neutral interface reflection, it is known
that highlights roughly correspond to the color of thelight source, making highlights particularly suited for
estimating the color of the light source [15, 16, 17].
However, detecting highlight pixels has proven to
be very challenging without prior knowledge of thescene [15, 18, 16, 19]. Edges, on the other hand, canautomatically be classified into different types without
much prior knowledge by using physics principles[20, 21, 22, 23]. For example, edges can be classified into material edges (e.g. edges between objects and object-background edges), shadow/shadingedges (e.g. edges caused by the shape or position ofan object with respect to the light source) and specular edges (i.e. highlights). These edges may have a
distinctive influence on the performance of illuminantestimation.
In this paper, the use of distinct edge types is
exploited to improve the performance of edge-based
color constancy by computing a weighted average ofthe edges. The weights are computed using a photometric edge classification scheme. Since such methods
often assume the scene is illuminated by a white lightsource, the automatic detection of such edges canbecome erroneous when the color of the light source is
not white. To this end, an iterative weighting scheme
is proposed that sequentially estimates the color of thelight source and updates the computed edge weights.
The rationale behind this approach is to fully exploit
the information that is enclosed in the image, andsimultaneously increase the accuracy of the illuminantestimation and (specular) edge detection.
This paper is organized as follows. In section 2,

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATE

color constancy is discussed, followed by the introduction of color constancy by edge weighting insection 3. Then, in section 4, the performance ofedge-based color constancy is analyzed with respect
to different edge types. Finally, in sections 5 and 6,
the method is evaluated and the obtained results are
discussed.

2

C OLOR C ONSTANCY

The image values f = (fR , fG , fB )T for a Lambertiansurface depend on the color of the light source I(λ),
the surface reflectance S(x, λ) and the camera sensitivity function ρ(λ) = (ρR (λ), ρG (λ), ρB (λ))T , where
λ is the wavelength of the light and x is the spatial
coordinate (e.g. [3, 10, 8]):Z
fc (x) = m(x) I(λ)ρc (λ)S(x, λ)dλ,(1)
ωwhere ω is the visible spectrum, m(x) is Lambertianshading and c = {R, G, B}. It is assumed that thescene is illuminated by uniform illumination and thatthe observed color of the light source e depends onthe color of the light source I(λ) as well as the camera
sensitivity function ρ(λ):
 ZeR
I(λ)ρ(λ)dλ.
(2)
e =  eG  =ωeBColor constancy can be achieved by estimating thecolor of the light source e, given the image values off , followed by a transformation of the original image
values using this illuminant estimate [24]:ft = Du,t fu ,(3)

where fu is the image taken under an unknown lightsource, ft is the same image transformed, so it appears
if it was taken under the canonical illuminant, andDu,t is a diagonal matrix which maps colors thatare taken under an unknown light source u to theircorresponding colors under the canonical illuminantc. The aim of this transformation is not to scale thebrightness level of the image, since the color constancy methods proposed and compared in this paper
only correct for the chromaticity of the light source.
Since both I(λ) and ρ(λ) are, in general, unknown,
the estimation of e is an under-constrained problem
that cannot be solved without further assumptions.
2.1

Pixel-based Color Constancy

Two well-known algorithms that are often used are
based on the Retinex Theory proposed by Land [7]:the White-Patch and the Grey-World algorithm. The
White-Patch algorithm is based on the White-Patch
assumption, i.e. the assumption that the maximum
response in the RGB-channels is caused by a white patch.
In practice, this assumption is alleviated by considering the color channels separately, resulting in the2

max-RGB algorithm. The Grey-World algorithm [5] isbased on the Grey-World assumption, i.e. the averagereflectance in a scene is achromatic. Another type of algorithms is gamut-based methods, originally proposedby Forsyth [10]. Gamut-based algorithms use moreadvanced statistical information about the image, andare based on the assumption, that in real-world images,
one observes, under a given illuminant, only a limited
number of different colors.
Another pixel-based algorithm, that is related tothe current paper, is proposed by Tan et al. [17].
This approach is based on the dichromatic reflection
model, and uses specularities or highlights for theestimation of the illuminant. By transforming theimage to inverse intensity chromaticity space, pixels are
identified that have a low body reflectance factor(effectively identifying pixels with a color that is closeor identical to the color of the light source). However,
the identification of such specular pixels remains a
problem.
2.2

Edge-based Color Constancy

Extending pixel-based methods to incorporate derivative information, i.e. edges and higher-order statistics,
resulted in the Grey-Edge [14] and the derivativebased Gamut mapping algorithms[13].
The Grey-Edge actually comprises a frameworkthat incorporates zeroth-order methods (e.g. the GreyWorld and the White-Patch algorithms), first-ordermethods (i.e. the Grey-Edge), as well as higher-ordermethods (e.g. 2nd -order Grey-Edge). Many different
algorithms can be created by varying the three parameters in:
 p1
Z
p
∂ n fc,σ (x)n,p,σdx= kec ,
(4)e=∂xnwhere | · | indicates the absolute value, c = {R, G, B},
n is the order of the derivative, p is the Minkowskinorm and k a multiplicative scalar constant chosensuch that the illuminant vector e has unit length.
Further, the derivative is defined as the convolutionof the image with the derivative of a Gaussian filter
with scale parameter σ [25]:∂ s+t Gσ
∂ s+t fc,σ= fc ∗st∂x y∂xs ∂y t(5)where ∗ denotes the convolution and s + t = n. Good
results are obtained by using instantiation e1,1,σ , i.e. a
simple average of the edges at scale σ also called theGrey-Edge method [14].
Another pixel-based method which has been extended to incorporate derivative information is the
Gamut mapping algorithm [13]. It can be proventhat linear combinations of image values also form
gamuts, thereby extending the Gamut mapping theory to incorporate image derivatives. In this paper,
we assess the influence of various edge types on theIEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATE

performance of both the Grey-Edge method and thederivative-based Gamut mapping method.

3

C OLOR C ONSTANCY BY E DGE W EIGHT-

ING

Consider the following simplified version of eq. (4):
 p1
Z
p
= kec ,
(6)
|fc,x (x)| dx
where fc,x (x) is the derivative of color channel c ∈{R, G, B} of image f at a certain scale. Then, theweighted Grey-Edge algorithm is given by:
Z
 p1
κ
p
|w(f ) fc,x (x)| dx= kec ,
(7)
where w(f ) is a weighting function that assigns aweight to every value of f and κ can be used to
enforce the weights (a higher value of κ results inmore emphasis on higher weights).
Numerous different weighting schemes can be incorporated in eq. (7), but in order to accurately estimate the color of the light source, a suitable weighting
scheme should enforce relevant information about
the color of the light source and disregard irrelevant
information. Since it is known that highlights area valuable cue for estimating the color of the lightsource [15, 16, 17], an obvious choice would be to compute weights using specular edge detection methods.
3.1

Edge Types and Classification

Various edge types are considered, i.e. material edges,(colored) shadow or shading edges, specular edges
and interreflection edges. Material edges are transitions between two different surfaces or objects. Shading edges are transitions that are caused by the geometry of an object, for instance by a change in surface
orientation with respect to the illumination. Shadow
edges are cast shadows, caused by an object that(partially) blocks the light source. Blocking of the lightsource often results in merely an intensity gradient,but sometimes a faint color gradient is introduced(provided two illuminants are present in a scene).
When we refer to shadow edges in general, both intensity and colored shadow edges are implied. Finally,in real-world images, interreflection is an important
aspect. Interreflection is the effect of light reflectedfrom one surface onto a second surface. This effectchanges the overall illumination that is received by
the second surface, and hence the color of this surface.
Finally, note that combinations of edge types (e.g. ashadow or shading edge that coincides with a materialedge) can also occur, but are not handled explicitly
here.
Generally, edge classification is based on photometric information. For instance, Finlayson et al. [20]propose to project a 2D log-chromaticity image onto3

the direction orthogonal to the light source, resulting
in a new image that is invariant to light intensity andcolor. Shadow edges are then detected by subtracting the derivatives of the invariant image and theoriginal image. Alternatively, Geusebroek et al. [21]propose a rule-based approach in combination witha set of color invariants derived from the KubelkaMunk theory for colorant layers. A slightly different
rule-based approach is proposed by van de Weijer
et al. [23], which is based on the same photometricinvariant principles. Geometric information is mostly
ignored for general edge classification methods, although some recent advancements show that shadowedge detection can benefit from geometric features[26, 27]. The weighted Grey-Edge method given byeq. (7) can incorporate classifications based on bothphotometric and geometric features, but in this paper
the focus will be mainly on photometric features.
More specifically, the quasi-invariants [23] are used
to design several different soft weighting schemes,
resulting in an elegant incorporation of weighting
schemes into edge-based color constancy.
3.2

Edge Weighting Schemes

Quasi-invariants [23] are computed using the derivative of an image, fx = (fR,x , fG,x , fB,x )T , and a set ofthree photometric variants. By removing the variancefrom the derivative of the image, a complementary
set of derivatives is constructed called quasi-invariants.
The edge energy contained in the three variant directions is indicative for the type of edge, e.g. if mostenergy is contained in the specular direction, the edge
is most likely to be a highlight. Using the quasiinvariants, three different weighting schemes can bederived (including all combinations of these schemes):
the specular weighting scheme, the shadow weighting
scheme and the material weighting scheme.
Specular Edge Weighting Scheme. The quasiinvariants decompose a derivative image into three
directions. The projection of the derivative on the
specular direction (i.e. the color of the light source)
is called the specular variant and is defined as:Ox = fx · ĉi ĉi ,(8)where ĉi = √13 (1, 1, 1)T is the color of the light source(assumed to be white here) and the dot indicates thevector inner product. The specular variant is that partof the derivative which could be caused by highlights.
What remains after subtraction of the variant from thederivative is called the specular quasi-invariant:
Otx = fx − Ox ,(9)

The quasi-invariant Otx only contains shadow-shading
and material edges, and is insensitive to highlightedges. Since all derivative energy of an image f iscontained in either of the three variant directions, theratio of the energy in the specular variant versus theIEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATE

total amount of derivative energy is an indication if
an edge is specular or not. This ratio translates to thefollowing specular weighting scheme:
ws,specular (fx ) =|Ox |,||fx ||(10)

where |Ox | is the absolute value of Ox and ||fx || =
q
2
2
2 .
fR,x
+ fG,x+ fB,x
Shadow Edge Weighting Scheme. Using the same
reasoning on the shadow-shading direction f̂ (i.e.
intensity), a shadow-shading invariant and quasiinvariant is obtained:Sx =fx · f̂ f̂ ,
(11)
Stx

=

fx − Sx ,(12)1where f̂ = √R2 +G(R, G, B)T . It can be derived
2 +B 2that the shadow-shading quasi-invariant is insensitive
to shadow edges. Translating this variant to a shadow
weighting scheme yields the following result:

ws,shadow (f ) =|Sx |.
||fx ||(13)

Material Edge Weighting Scheme. Finally, theshadow-shading-specular variant and quasi-invariant
can be constructed by projecting the derivative on thehue direction:
Hx =fx · b̂ b̂,
(14)
Htx

= fx − H x ,(15)where b̂ is the hue direction, which is perpendicular
to the previous two directions:b̂=f̂ × ĉi.

f̂ × ĉi(16)

The resulting quasi-invariant Htx does not containspecular or shadow-shading edges, and can be used
to assign higher weights to material edges as follows:
ws,material (fx ) =|Htx |.
||fx ||(17)

To evaluate the influence of the edge-type classifieron the color constancy results, one additional experiment in section 5 will make use of a different specularweighting scheme, based on geometric features similar to [26].
3.3

Iterative Weighted Grey-Edge

In equations (8) and (16), it can be derived that the
specular and shadow-shading-specular variants andquasi-invariants are dependent on the color of thelight source (the shadow-shading variant and quasiinvariant are not). The underlying assumption thatthe scene is viewed under a white (or neutral) lightsource [23] is obviously not met for the images in
the used data sets prior to applying color constancy.

4

However, after the proposed algorithm is applied,
the illuminant should be neutral, at least in theory.
Hence, we propose to first correct the input image
A with an estimated illuminant I. Then, using thiscolor corrected image B, we can compute a weighting
scheme W, which in turn is used by the weighted GreyEdge algorithm to compute an updated estimate of
the illuminant I. After some iterations, the illuminantestimate will approximate a white light source, atwhich point the accuracy will no longer increase
and the method has converged. Consequently, we
propose to iteratively apply the weighted Grey-Edgealgorithm, where a new instantiation of the weighting
scheme is computed every iteration based on thecolor corrected image at each iteration. The iterative
weighted Grey-Edge is given by the following linesof pseudo-code:
Algorithm 1 Iterative Weighted Grey-Edge
Input:
input image: A
initial illuminant estimate: I
stopping criterion: C
Method:while (¬ C) do
B = color_correct(A, I)
W = compute_weighting_scheme(B)I’ = weighted_GreyEdge(B,W)
I = I ∗ I’
C = update_stopping_criterion()end while
For sake of clarity, we will not change the type of
weighting scheme W (e.g. specular or shadow-based)throughout the iterations. Further, the initial illuminant estimate I can either be a white light source((1, 1, 1)T ) or it can be the result of any color constancy
algorithm. Finally, the stopping criterion C can be
defined as a fixed number of iterations, or as somemeasure of convergence (e.g. the distance between awhite light source and the illuminant I’ at the endof each iteration is below some threshold).

4 P ERFORMANCE USING D IFFERENT E DGE
T YPESIn this section, the aim is to analyze which edge
types have the most influence on the accuracy of theilluminant estimation. To this end, a spectral data
set is used first to generate different edges types
under controlled circumstances. On this data set, thetwo different edge-based color constancy algorithms,
i.e. the Grey-Edge and the derivative-based Gamut
mapping approach, are evaluated.
To evaluate the performance of color constancy algorithms, the angular error  is widely used [28]. Thismeasure is defined as the angular distance between

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATE

5

the actual color of the light source el and the estimatedcolor ee :
 = cos−1 (êl · êe ),(18)

where êl · êe is the dot product of the two normalized
vectors representing the true color of the light source
el and the estimated color of the light source ee . To
measure the performance of an algorithm on a whole
data set, the median angular error is reported.
4.1

Spectral data

The first experiments are performed using the spectral
data set introduced by Barnard et al. [29]. This set
consists of 1995 surface reflectance spectra and 287
illuminant spectra, from which an extensive range of
surfaces (i.e. RGB-values) can be generated using eq.
(1). As discussed before, for these experiments, thefollowing types of surfaces are created:• Material surface Zmi :mik =Ik (λ)ρ(λ)Si (x, λ)dλ.

(19)

ω
•

•

Intensity shadow
Z surface pi :Ik (λ)
ρ(λ)Si (x, λ)dλ.
pik =
φ
ω

(20)

Colored shadow surface
qi :Z
qikk0 = pik + η

Ik0 (λ)ρ(λ)Si (x, λ)dλ,

(21)

ω
•

Specular surface hik :Z
hik = mik + γ

Ik (λ)ρ(λ)dλ,Fig. 1. Median angular error of the Grey-Edge, top
figure, and the Derivative-based Gamut mapping, bottom figure, including a 95% confidence interval, using
several different edge types.

(22)

ω
•

Interreflection surface
rijk :
Z
rijk =

Ik? ρ(λ)Si (x, λ)dλ,

(23)

ω

where Ik? = Ik (λ) + θIk (λ)Sj (x, λ). Further, the subscripts i and j denote different surface reflectancespectra, k and k 0 denote different illuminant spectra.
The random variables φ and γ are uniformly distributed between 1 and 4, and η and θ are randomvariables uniformly distributed between 0 and 0.25.
Since the focus is on edge-based color constancy,
the following transitions (i.e. edges) between surfaces
are generated:• Material edge: mik − mjk .
• Intensity shadow edge: mik − pik .
• Colored shadow edge: mik − qikk0 .
• Specular edge: mik − hik .
• Interreflection edge: mik − rijk .
A material edge is generated by taking the difference
between two different material surfaces, mi − mj .
The difference between a material surface mi and thesame surface under a weaker light source results in
an intensity shadow edge, mi − pi . A colored shadowedge is defined as the difference between a material
surface mi and a colored shadow surface, mi − qi .
A specular edge is defined as the difference between

a material surface mi and the bright version of thesame material, mi −hi . Finally, an interreflection edge
is defined as the difference between a material surface
mi and an interreflection surface rij where surface miinterreflects onto a second surface mj , hence mi − rij .
Note that these edges can be considered to be stepedges. In real-world scenes, transitions are likely to
be more gradual. However, for the purpose of theanalysis performed in sections 4.2 and 4.3, these edges
are used to give a best-case relative assessment ofalgorithm performance, comparing the different edge
types under the same conditions. Further, we would
like to note that the intrinsic properties of the useddata set (i.e. the average of all surfaces is not grey) isa potential cause for error, but to avoid confusion wewill ignore this in the remainder of this section.
4.2

Different number of edges

In the first experiment, the performance of two edgebased color constancy algorithms is analyzed withrespect to different edge types. Using the spectral dataset, a number of random surfaces are created, including n material surfaces, n intensity shadow surfaces,n colored shadow surfaces, n specular surfaces andn interreflection surfaces, resulting in a total of 5n

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATE

(a)

(b)

(c)

6

(d)

(e)

Fig. 2. Gamut in opponent color space of several edge types put under one illuminant which is specified by the
fourth axis. Shown are material edges in figure (a); intensity shadow edges in figure (b); colored shadow edges
in figure (c); specular edges in figure (d); interreflection edges in figure (e).

surfaces. Note that to create these surfaces, the same
illuminant is used. Using these surfaces, n material
edges, n intensity shadow edges, n colored shadow
edges, n specular edges and n interreflection edges
are created. Two edge-based color constancy algorithms are evaluated (the Grey-Edge algorithm andthe derivative-based Gamut mapping) by gradually
increasing the number of edges. For each value ofn (n = {4, 8, 16, 32, 64, 128, 256, 512, 1024}), the experiment is repeated 1000 times.
In figure 1 (top graph), the median angular error
for the Grey-Edge algorithm is shown differentiatedby these five edge types. The angular error whenusing material edges is significantly higher than whenusing intensity shadow edges. As expected, color
constancy based on specular edges results in a close toideal performance. Further, the performance using thecolored shadow edges and the interreflection edges
is similar to the performance when using the material edges. The performance of the derivative-based
Gamut mapping, see figure 1 (bottom graph), showsa similar trend. Using specular edges results in nearperfect color constancy, and intensity shadow edges
are more favorable than the three other types of edges.

two material surfaces) is computed as:Z
mik − mjk =Ik (λ)ρ(λ)(Si (λ) − Sj (λ))dλ.

(25)

ω

Substituting eq. (25) into eq. (24) results in the following underlying assumption:
N

1 X
Rand (|Si (λ) − Sj (λ)|) = a.
i,jN →∞ N
1lim(26)

Under the assumption that the surface reflectances arenormally distributed with mean µ and variance σ 2 ,then subtracting two surfaces results in a new randomvariable with larger variance (2σ 2 ). On the other hand,consider the assumption of (intensity) shadow edgebased color constancy we have:
N

1 X
lim
Rand (|mik − pik |) = ai
N →∞ N1Z
Ik (λ)ρ(λ)dλ, (27)ω

where pik is an intensity shadow surface as definedby eq. (20). Substituting eq. (20) into eq. (27) resultsin the following underlying assumption:
N

4.3

Gamuts of different edge types

To analyze the high error for the material edges andto explain the differences in performance of the othertypes of edges, we will go into detail on the underlying assumptions. First, consider the assumption ofmaterial edge-based color constancy:

1 X
Rand (|Si (λ)|) = a.
i
N →∞ N1lim(28)

Under the same assumption that the surface reflectances are normally distributed with mean µ andvariance σ 2 , it can be observed that the variance of theintensity shadow edges (σ 2 ) is lower than the varianceof the material edges (σ 2 < 2σ 2 ). This implies that aZ
NX
1larger number of (different) material edges is required
lim
Rand (|mik − mjk |) = a Ik (λ)ρ(λ)dλ, (24)i,j
N →∞ Nto obtain the same accuracy as shadow-edge-basedω1color constancy.
where mik is a material surface as defined by eq. (19),This analytical derivation is supported by the emN is the number of edges, Randi,j (.) is a function that pirical distribution of these two edge types. For the
randomly selects two surfaces i and j and a is a scalar ease of illustration of the physical properties of edge
value. Further, a material edge (the difference between types, the edges are converted to the opponent color

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATE

7

Fig. 3. Mean angular error using material edges, shadow edges and specular edges, for different clipping values.

space:
O1x

=

O2x

=

O3x

=

Rx − Gx√,2
Rx + Gx − 2Bx√,
6Rx + Gx + Bx
√
3

(29)
(30)
(31)

where Rx , Gx and Bx are derivatives of the R, G and
B channels, respectively.
The distribution of edge responses in the opponent
color space is shown in figure 2. From these graphs,
it can be derived that the variation in edge color ismuch higher for the material edges, figure 2(a), thanfor shadow edges, figures 2(b) and (c), which is also
analytically derived. Further, the intensity shadow
edges are more directed towards the color of the lightsource (shown by the fourth axis) than the coloredshadow edges. The shape of the gamut of the color
shadow edges, which appears to be less directed
towards the color of the light source than other edge
types, can be explained by the influence of the second
light source. The gamut of interreflection edges, figure2(e), is similar to the material edges. Finally, specularedges, figure 2(d), all align perfectly with the color of
the light source (shown by the fourth axis).
These graphs show that it is beneficial to use edges
that are aligned with the color of the light source.
The specular edges are all distributed on the diagonal
representing the color of the light source, and nearperfect color constancy can be obtained using theseedges. This observation is in accordance to pixel-based
highlight analysis, where highlights contain valuable
information about the color of the light source [15, 16].
Shadow edges are distributed denser around the colorof the light source than material edges and interreflection edges, resulting in a higher performance.
Color clipping. In practice, pixel values are often
bound to a certain maximum value. This effect iscalled color clipping. Since the specular surfaces havethe highest RGB-values, these surfaces (and consequently the specular edges) risk to be affected by color
clipping. To analyze this effect, a second experimentis performed where the generated RGB-values arecolor clipped at a gradually decreasing value. The
results of this experiment for the Grey-Edge algorithm
are shown in figure 3. The derivative-based Gamut
mapping reveals a similar trend (not shown here). The
performance using the specular edges immediately
starts to decrease significantly. The performance using
the material and the shadow edges is less affected; theangular error does not significantly increase until 40%of the total number of surfaces are clipped. The effectsof color clipping cause the gamuts of the specularedges to shift towards the intensity axis (O3x ), hence
the estimate of the illuminant will be biased towards
white. Color clipping is an often occurring phenomena and cannot be prevented in practice. To alleviate
the effects of color clipping on the performance of any
color constancy algorithm, pixels that are potentiallycolor clipped are often discarded. Practically, thismeans that pixels with a maximum response in eitherof the three color channels are not considered while
estimating the illuminant.
To conclude, from an analytical approach, it canbe derived that using specular edges for edge-based
color constancy results in a close to ideal performance, because the specular edges align with thecolor of the light source. However, in practice, color
clipping may eliminate the advantages of specularedges and cause a decrease in performance. Shadow
edges contain more variation than specular edges but
are still aligned with the color of the light source.
Consequently, the performance of edge-based color
constancy using shadow edges degrades only slightly
with respect to using highlights. However, as material
edges vary even more, their performance degrades
even more. Although interreflection edges vary lessthan material edges, they are not aligned with thecolor of the light and hence their performance is theworst.

5

E XPERIMENTS

Experiments using the (iterative) weighted Grey-Edgemethod are performed on several data sets. First,
experiments are performed on a data set containing
indoor images that are recorded under controlled
settings, see section 5.1. Then, in section 5.2, results areIEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATE8

All 321 images

Mean 

Median Max Mean 

Median Max Mean 

Median Max 

All 321 images

ws,specular
ws,shadow
ws,material

9.0◦
5.5◦
19.3◦

5.6◦
3.1◦
17.1◦

43.3◦
33.3◦
56.0◦

5.6◦
5.5◦
28.5◦

2.4◦
3.3◦
26.4◦

43.8◦
33.1◦
78.3◦

3.4◦
5.6◦
13.2◦

2.1◦
3.3◦
12.3◦

25.0◦
37.6◦
37.6◦

ws,specular
ws,shadow
ws,material

Subset A

Mean 

Median Max Mean 

Median Max Mean 

Median Max 

Subset A

ws,specular
ws,shadow
ws,material

8.9◦
4.8◦
19.2◦

5.9◦
3.0◦
16.4◦

43.3◦
24.5◦
56.0◦

5.1◦
4.7◦
30.5◦

2.0◦
3.0◦
28.3◦

43.8◦
23.6◦
78.3◦

3.4◦
4.9◦
12.8◦

1.8◦
3.1◦
11.3◦

25.0◦
24.5◦
37.6◦

ws,specular
ws,shadow
ws,material

Subset B

Mean 

Median Max Mean 

Median Max Mean 

Median Max 

Subset B

ws,specular
ws,shadow
ws,material

9.2◦
7.0◦
19.5◦

5.6◦
3.6◦
18.1◦

31.2◦
33.3◦
50.2◦

6.8◦
7.1◦
24.0◦

3.3◦
3.6◦
20.6◦

35.1◦
33.1◦
70.4◦

5.3◦
7.2◦
14.2◦

3.2◦
3.7◦
14.8◦

23.4◦
33.3◦
32.6◦

ws,specular
ws,shadow
ws,material(a) Performance after one iteration(b) Performance after 10 iterations(c) Performance of theoretical scenario

TABLE 1
Mean, median and maximum angular errors on the SFU Controlled Indoor set. All results in this table are
obtained using the optimal parameter settings for the proposed iterative weighted Grey-Edge (i.e.
Minkowski-norm = 7, σ = 5 and κ = 8). Table (a) shows the results of the weighted Grey-Edge for threedifferent weighting schemes, table (b) shows the results of applying the iterative weighted Grey-Edge for thesame three weighting schemes, and table (c) shows the results of the theoretical scenario, where we computethe weighting schemes from the color corrected images.

reported on two unconstrained data sets with natural
images.
5.1 SFU Controlled Indoor
The first data set, denoted by SFU Controlled Indoor[29] consists of 31 different scenes, recorded under
11 different light sources, resulting in a total of 321
images. Two relevant subsets are distinguished todemonstrate the robustness of the proposed algorithm; one subset contains 223 images with minimal specularities (denoted subset A), another subset contains 98 images with non-negligible di-electric
specularities (denoted subset B). The main difference
between these two subsets is the fact that the images
in subset A (some of which are flat mondrian-likecompositions of colored paper) contain few or no
specularities, while all images in subset B contain atleast some highlights. Some examples are shown infigure 6 (top row).
Weighting Schemes. Results of applying the threeweighting schemes, emphasizing specular, shadow ormaterial edges, are shown in table 1. Interestingly,
results of assigning higher weights to specular edges
initially, i.e. after one iteration, is worse than assigninghigher weights to shadow edges. The reason for this is
explained in section 3, i.e. the specular quasi-invariant
assumes a neutral illumination while the shadowshading quasi-invariant does not. Hence, the detectionof highlights suffers from the colored light sources.
Running multiple iterations increases the accuracyof the specular edge detection, and hence the accuracy of the weighted Grey-Edge using the specularweighting scheme, see table 1(b). The effects of therunning multiple iterations has only minor effects on
the shadow weighting scheme, while the performanceof the material weighting scheme considerably deteriorates, see also figure 4(a). The latter can be explainedby the misclassification rate of the specular edges:
when running multiple iterations, less specular edges
are misclassified as material edges. The fact that theperformance of the shadow weighting scheme is notaffected by running multiple iterations is expected
because the shadow-shading quasi-invariant is robust
to illumination changes (see section 3.2).
Results shown in table 1 are obtained using a
relatively high value for κ (κ = 8). The effect ofusing lower values for κ is shown in figure 4(b). It
can be observed that especially the specular weighting
scheme benefits from an increased value for κ, while
the shadow weighting scheme is not affected at all.
Influence of Edge Classification Accuracy. The
results of the iterative weighted Grey-Edge show thatthe proposed method benefits from accurate specularedge detection. An illustration is presented in figure 5,
showing the results and corresponding weight maps
for two individual images of the first, second andfinal iteration. It can be seen that the accuracy ofthe edge classification and the illuminant estimates
increase simultaneously. These examples imply thatthe proposed method can benefit from accurate specular edge detection, while specular edges can be deClassifier
SIFT + Q.I.
Quasi-Invariants
Sub-optimal Q.I.

AUC
0.83
0.78
0.68

Mean 

Median 

Max 

5.2◦
5.6◦
5.6◦

2.6◦
2.9◦
3.2◦

28.5◦
32.6◦
32.5◦

TABLE 2
Results of using weighting schemes computed byapplying different specular edge detection classifierson the SFU Controlled Indoor set. The area under the
ROC-curve (AUC) values are determined using
cross-validation on the training set.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATE

(a)

(b)

Fig. 4. The effects of applying several iterations to
the different weighting schemes in figure (a). Figure(b) shows the effects of increasing the value of κ fordifferent weighting schemes. Results are obtained on
the SFU Controlled Indoor set.

tected more accurately on images with more accurate
color constancy. To further confirm that the improved
performance of the iterative weighted Grey-Edge isdirectly related to the accuracy of the specular edge
detection, two additional experiments are performed.
As the quasi-invariants are based on neutral illumination, computing the specular weight map on theimages color corrected with the ground truth would
result in the highest edge detection accuracy. Using
this weight map together with the original uncorrected
images to estimate the color of the light source will
give an indication of the potential of the proposed
method. The results of this (theoretical) experimentare shown in table 1(c), where it can be observed that
especially the performance of the specular weighting
scheme can benefit from even more accurate specularedge detection.
Influence of Different Edge Classifiers. For the
next experiment, the system proposed in [26] isadapted to specular edge detection. Patches fromthe same data set as used in [26] are manually selected and annotated. Note that these patches haveno overlap with the color constancy data set. Using
features derived from these patches, a classifier (SVM)
is learned that is able to distinguish patches containing specular edges from patches without specularedges. By applying this classifier to a full image using
a sliding window approach, a posterior probability
is obtained for every block of pixels in the image.
Finally, a smoothing filter is applied to reduce the inherent uncertainty of the block-based detection result.
These smoothed posterior probabilities are directly
used as weights in eq. (7).
Three different classifiers are learned, each with adifferent accuracy. The accuracy is measured as the9

area under the ROC-curve (AUC), and is determinedusing cross-validation on the training patches. The
first classifier uses the SIFT-feature in combination
with the quasi-invariants (AUC = 0.83), the secondclassifier uses merely the quasi-invariants (AUC =0.78) and the third classifier uses the quasi-invariants
with sub-optimal SVM-parameters (AUC = 0.68). Results of the weight maps computed using these threeclassifiers are summarized in table 2. It can be seenthat the classifier with the highest accuracy, i.e. thecombination of SIFT and the quasi-invariants, resultsin the best color constancy performance. Moreover,the median angular errors of the sub-optimal classifier
using the quasi-invariant features is outperformed by
the optimal classifier using the same features.
Finally, to verify whether the edge detection accuracy of the training set corresponds to the accuracy on
the test images, we manually annotated highlights in
the SFU Controlled Indoor set. Using these manuallylabelled highlights and the output of the three classifiers, we are able to relate the classifier performanceto the color constancy output. The accuracy of theclassifier is dependent on the threshold on the classifier posterior probability, e.g. a low threshold willclassify many edges as highlight and consequently
result in a high recall but low precision, while a highFig. 5. Two results of running multiple iterations of the
specular-weighted Grey-Edge. The first and second
images show the result after the first and second
iteration, the third image shows the result after thealgorithm converged. The weight maps used in the corresponding iterations are color coded, such that dark
red indicates a high weight and dark blue indicates a
low weight.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATE

MethodGrey-World (e0,1,0 )White-Patch (e0,∞,0 )Using I.I.C. Space
Gamut mapping (σ = 4)
general Grey-World (cross-val.)
1st -order Grey-Edge (cross-val.)
2nd -order Grey-Edge (cross-val.)
Proposed: Iterative (cross-val.)

Mean 

Median 

Max 

9.8◦
9.1◦
15.5◦
4.6◦
5.4◦
5.7◦
5.3◦
5.6◦

7.0◦
6.5◦
8.2◦
3.3◦
3.5◦
3.4◦
2.9◦
2.9◦

37.3◦
36.2◦
80.9◦
27.1◦
28.8◦
31.8◦
27.6◦
46.7◦

TABLE 3Comparison to other algorithms on the SFU
Controlled Indoor. Note that cross-validation results
are obtained using three-fold cross-validation (thereported results are averaged over 100 repetitions).

threshold will classify few edges as highlight resulting
in a low recall but high precision. The F-measure
is often used to summarize the trade-off between
precision and recall, so we use this measure here. For
each method we select the threshold which yields thehighest F-measure. The results of the three classifiers
are 0.960 (SIFT-feature in combination with the quasiinvariants), 0.729 (quasi-invariants) and 0.513 (quasiinvariants with sub-optimal SVM-parameters). These
results are in accordance with the color constancy
performance in table 2 and confirm that the proposed
method benefits from more accurate specular edge
detection.
Comparison to State-of-the-Art. Results of stateof-the-art methods are shown in table 3. It should
be noted that the proposed method using specularweights, as well as several other state-of-the-art methods contain parameters that considerably influence
the results. Therefore, we report the performance ofapplying cross-validation to determine the parametersettings automatically (denoted cross-val. in table 3).
The automatic selection of parameters is performed bythree-fold cross-validation, where the reported results
are the average of 100 runs.
It can be derived that the proposed method is comparable to state-of-the-art. For the gamut mapping, theimplementation of [13] is used. All images recorded
under one light source (syl-50MR16Q) are used toconstruct the canonical gamut, where we make surethat when testing images from a particular scene (e.g.
ball), the corresponding training image of that scene(e.g. ball under syl-50MR16Q) is not used for computation of the canonical gamut. The mean angularerror of the proposed method is slightly higher than
the gamut mapping and the general Grey-World, but
the median is considerably lower. Further, using the
Wilcoxon sign test at 95% confidence level[28] (results
are not visualized here), it is found that the proposed
method is significantly better than all methods except the the general Grey-World. Finally, we would
like to note that the proposed method outperforms
the unweighted Grey-Edge for all choices for σ and10

Minkowski-norm, provided the appropriate value forκ is selected.
5.2

Uncontrolled data sets

The next experiments are performed on two uncontrolled data sets containing a variety of scenes. The
first data set, denoted SFU grey-ball, consists of 15 clips
with a total of 11, 346 images [30]. These images are
stored in a non-linear device-RGB color space (NTSCRGB), so to create linear images we applied gammacorrection with γ = 2.2 and recomputed the groundtruth using these linear images. Further, since thecorrelation among the images is rather high, videobased analysis was applied to decorrelate the visualcontent of the data set [31], resulting in a smaller
but uncorrelated data set containing 1135 images (thereported results are obtained on the test set, consistingof 70% of the uncorrelated data, as indicated by [31]).
The second data set, denoted Color-checker, contains
568 images [32]. This data set uses a Macbeth Color
Checker that is carefully placed within the scene tocapture the ground truth. Note that the latter of thetwo data sets does not need additional processing toacquire linear images. Examples of these data sets areshown in figure 6 (middle and bottom rows).
The results on these two data sets mostly agreewith the previous experiments (see table 4). The error when using specular-based weights decreases as
the algorithm is applied multiple iterations, while
the error when using shadow-based weights roughly
remains stable. Although the increase in performanceon the SFU grey-ball set is only marginal with respect
to the unweighted Grey-Edge (a decrease in mean
angular error of approximately 2% is not perceptuallysignificant [33]), the proposed method still performs
significantly better than all other methods according
to the Wilcoxon sign test (at the 95% confidence level).
An overview of all results is shown in table 5. A possi-

Fig. 6. Some example images of the used data sets.
The top row shows some examples of the SFU Controlled Indoor set, the middle row shows examples of
the color checker set and the bottom row shows someexamples of the SFU grey-ball set.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATE11

SFU grey-ballMean 

Median Max Mean 

Median Max Mean 

Median 

Max 

SFU grey-ball

ws,specular
ws,shadow
ws,material

10.8◦
10.6◦
19.5◦

9.4◦
8.6◦
18.0◦

36.1◦
52.3◦
64.5◦

10.4◦
10.5◦
29.5◦

8.9◦
8.6◦
29.9◦

50.4◦
50.1◦
74.1◦

7.7◦
11.1◦
13.7◦

6.2◦
9.3◦
12.2◦

30.0◦
30.0◦
43.2◦

ws,specular
ws,shadow
ws,material

Color-checker

Mean 

Median Max Mean 

Median Max Mean 

Median Max Color-checker

ws,specular
ws,shadow
ws,material

7.6◦
7.0◦
19.6◦

5.5◦
5.2◦
19.1◦

36.8◦
35.1◦
49.4◦

6.5◦
7.0◦
37.7◦

4.7◦
5.2◦
37.5◦

44.◦
35.1◦
67.8◦

3.9◦
7.3◦
16.5◦

3.2◦
5.6◦
15.9◦

19.0◦
36.0◦
38.9◦

ws,specular
ws,shadow
ws,material(a) Performance after one iteration(b) Performance after 10 iterations(c) Performance of theoretical scenario

TABLE 4
Mean, median and maximum angular errors on the two Uncontrolled data sets. The results on the SFU
grey-ball set are obtained with κ = 10, the results on the color-checker set with κ = 50 (both sets are processedwith σ = 1).

ble reason for the apparent lack of improvement withrespect to the unweighted Grey-Edge on this data set
is the number of color clipped pixels: on average, 5%of the pixels in this data set are possibly clipped (apixel is possibly clipped if it has a maximum response
in either of the three channels). The color-checker set,
for instance, only consists of 0.5% color clipped pixels.
Since these pixels are ignored during the estimationof the light source, they can not contribute to moreaccurate estimations. In fact, as was shown in section
4, the accuracy will rapidly degrade if a significant
percentage of the pixels with high intensity (likely
specular pixels) is ignored.
Applying the proposed method to the color checker
set results in state-of-the-art performance, see table 6.
The proposed method significantly outperforms theother methods according to the Wilcoxon sign test,
including the gamut mapping. It should be noted that
the Wilcoxon sign test is based on the number ofimages on which one method performs better thananother. In the comparison between the proposed
method and the gamut mapping, it was found thatthe proposed method performs better than the gamut
mapping on the majority of the images, resulting
in a statistical significant difference in favor of the
proposed method. Running multiple iterations using
the specular weighting scheme improves the performance considerably: compared to the unweighted
Grey-Edge, a decrease in angular error of more than

MethodGrey-World (e0,1,0 )White-Patch (e0,∞,0 )Using I.I.C. Space
Gamut mapping (σ = 7)
general Grey-World (cross-val.)
1st -order Grey-Edge (cross-val.)
2nd -order Grey-Edge (cross-val.)
Proposed: Iterative (cross-val.)

Mean 

Median 

Max 

13.0◦
12.3◦
14.0◦
11.9◦
11.3◦
10.7◦
10.8◦
10.4◦

10.6◦
10.0◦
10.6◦
9.0◦
9.8◦
9.1◦
9.2◦
9.0◦

55.2◦
38.7◦
72.0◦
39.1◦
42.3◦
48.5◦
46.5◦
50.1◦

TABLE 5Comparison to other algorithms on the SFU grey-ball
set.

11% can be obtained. This difference is perceptuallyand statistically significant.
Some example results of both the SFU grey-ball andthe color-checker set are shown in figure 7. Overall,
it can be concluded that the proposed method using
specular edges improves upon the traditional GreyEdge method. Assigning higher weights to specularedges can lead to an improvement of up to 11% (onthe color-checker set). However, from the experimentsin this section, it becomes clear that the proposed
method introduces stronger (but not more) outliers:
the maximum angular error increases on all data sets,
but the median angular error decreases.

6

D ISCUSSION

In this paper, it is shown that weighted edge-based
color constancy based on specular edges can significantly improve unweighed edge-based color constancy. Further, it is shown that shadow edges contain valuable information. The reason for these conclusions can be derived as follows. Edges that are
achromatic when viewed under a white light source,
will accumulate in a tight gamut and assume thecolor of the light source when observed under coloredillumination. This will increase the saturation from 0
to the saturation of the light source. Consequently,
these edges are well suited for estimating the color of
the light source, as all properties of the light source
MethodGrey-World (e0,1,0 )White-Patch (e0,∞,0 )Using I.I.C. Space
Gamut mapping (σ = 3)
general Grey-World (cross-val.)
1st -order Grey-Edge (cross-val.)
2nd -order Grey-Edge (cross-val.)
Proposed: Iterative (cross-val.)

Mean 

Median 

Max 

9.8◦
8.2◦
9.7◦
7.1◦
7.1◦
7.1◦
7.3◦
6.7◦

7.4◦
6.1◦
6.0◦
4.9◦
5.4◦
5.6◦
5.5◦
5.0◦

46.0◦
36.3◦
61.3◦
37.0◦
36.7◦
36.3◦
37.3◦
42.1◦

TABLE 6Comparison to other algorithms on the Color Checker
set.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATE

12

Fig. 7. Some example results of several methods, applied to the uncontrolled data sets. Note that these images
are gamma-corrected for better visualization, but the estimation and correction are performed on the linear
images. The value reported in the bottom right corner indicates the angular error. The top two rows show
examples of the SFU grey-ball set, with from left to right, the original image, the result of correction with theground truth, the proposed method, the 1st -order Grey-Edge, the Grey-World and using I.I.C. Space. The bottom
two rows show examples of these algorithms applied to the color-checker set.

are contained in this edge. This was shown in section
4.
Colored edges, on the other hand, are edges that
correspond to the transition from one surface (e.g. a
red surface when viewed under a white light source)
to another surface (e.g. a blue surface when viewed
under a white light source). The saturation of this redto-blue edge when viewed under a white light source
can be an arbitrary value. Moreover, when this edge isobserved under a colored light source, the saturationand color will result in gamuts with large variationthat can take on unpredictable values, from which it
is extremely hard (if not impossible) to estimate thecolor of the light source. In general, the hue of an edge
is more altered by the illuminant when the saturationof that edge under a white light source is lower. Moreformally, a negative correlation exists between Sw and
dwu , where Sw is the saturation of an edge under awhite light source and dwu is the distance of that edge
under a white light source w to that same edge viewed
under an unknown light source u.
From this, it can be concluded that edges thatare unsaturated under a white light source are goodcandidates for estimating the color of the light source.
Specular and shadow edges are examples of suchedges. However, specular edges are difficult to detectbecause of disturbances like a colored illumination.
Using a specular edge detector that is dependent
on the color of the illumination will introduce thenecessity of running multiple iterations. The shadow

edges can be detected regardless of the color of thelight source, but are of less value than the specularedges.
If we assume highlights are the most important cue
for the estimation of the color of the light source, it
becomes obvious that Grey-Edge-based methods gain
more from this knowledge than pixel-based methods.
In a typical scene, the number of pixels that coincidewith the illuminant direction is vastly outnumberedby the number of pixels that do not coincide withthis direction: most pixels are not highlights or perfectreflectances. For edges, a similar conclusion holds,
although the ratio of edges that coincide with theilluminant direction versus edges that do not is likely
to be larger. Indeed, uniformly colored patches will
only have non-zero edge energy near the boundaryof the patch, so small patches will generate relatively more edges with non-zero energy than large
patches. Consequently, as highlights (or other pixels
that coincide with the color of the light source) areoften small regions in an image, the edges that arecaused by these regions are likely to stand out among
the other edges more than highlight pixels among
other pixels. Note, however, even though edges are
more likely to coincide with the illuminant direction
than pixels, this does not mean pixels are inferior toedges at all times. Moreover, even scenes with few or
no edges or pixels that coincide with the illuminantdirection could result in accurate illuminant estimates:
the average of a scene can still be an accurate cue

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATE

for the color of the light source if there is enoughvariation among the pixels or edges.
The main advantage of the proposed (iterative)
weighted Grey-Edge over existing methods is that
additional information, which is provided by the different edge types, is used. This information, whenavailable, results in more accurate illuminant estimates. Moreover, the proposed method connects twotheories involving color image analysis, i.e. the GreyEdge for color constancy and the Quasi-Invariants foredge classification. The disadvantage of the proposed
method is that in case of misclassification of the edge
types the method may result in lesser performance.
The weighted Grey-Edge inherits the weakness of theregular Grey-Edge: opaque parameter selection, i.e.
the optimal parameters are difficult to select without
prior knowledge of the input images.

7

C ONCLUSION

In this paper, the influence of different edge types
on the performance of edge-based color constancy isanalyzed.
It is shown that weighted edge-based color constancy based on specular edges can result in accurate
illuminant estimates. Using a weight map that isbased on shadow edges performs slightly worse than
specular edges, but considerably better than usingmaterial edges. However, the accuracy of the detection of specular edges is degraded by failing assumptions of the quasi-invariants (like the assumption ofa white light source). Iteratively classifying specularedges and estimating the illuminant can considerably
reduce the dependency of specular weights on the
intrinsic assumption of a white light source, and hence
result in a better performance. Weight maps that put
more emphasis on shadow edges are not dependent
on the color of the light source, and result in stable
illuminant estimates.
Experiments on images that are recorded undercontrolled circumstances demonstrate that the proposed iterative weighted Grey-Edge algorithm basedon highlights reduces the median angular error withapproximately 25%. Further, in experiments on images that are recorded in an uncontrolled environment, improvements in angular error up to 25% withrespect to unweighted edge-based color constancy are
obtained.

ACKNOWLEDGMENT
This work has been supported by the EU projects
ERGTS-VICI-224737; by the Spanish Research
Program
Consolider-Ingenio
2010:
MIPRCV(CSD200700018); and by the Spanish projects
TIN2009-14173. Joost van de Weijer acknowledges
the sup- port of a Ramon y Cajal fellowship.

13

R EFERENCES[1]

T. Gevers and A. Smeulders, “Pictoseek: combining color and shape invariant features for image
retrieval,” IEEE TIP, vol. 9, no. 1, pp. 102–119,
2000.
[2] M. Fairchild, Color Appearance Models, 2nd Ed.
Chichester, UK: John Wiley & sons, 2005.
[3] S. Hordley, “Scene illuminant estimation: past,present, and future,” CRA, vol. 31, no. 4, pp. 303–
314, 2006.
[4] M. Ebner, Color constancy, ser. Wiley-IS&T Series
in Imaging Science and Technology. John Wiley& Sons, 2007.
[5] G. Buchsbaum, “A spatial processor model for
object colour perception,” J. Franklin Institute, vol.
310, no. 1, pp. 1–26, 1980.
[6] G. Finlayson and E. Trezzi, “Shades of gray andcolour constancy,” in Proc. CIC, 2004, pp. 37–41.
[7] E. Land, “The retinex theory of color vision,”
Scientific American, vol. 237, no. 6, pp. 108–128,
December 1977.
[8] G. Finlayson, S. Hordley, and P. Hubel, “Colorby correlation: a simple, unifying framework forcolor constancy,” IEEE TPAMI, vol. 23, no. 11, pp.
1209–1221, 2001.
[9] G. Finlayson, S. Hordley, and I. Tastl, “Gamut
constrained illuminant estimation,” IJCV, vol. 67,
no. 1, pp. 93–109, 2006.
[10] D. Forsyth, “A novel algorithm for color constancy,” IJCV, vol. 5, no. 1, pp. 5–36, 1990.
[11] A. Chakrabarti, K. Hirakawa, and T. Zickler,“Color constancy beyod bags of pixels,” in Proc.
of the IEEE Conf. on Computer Vision and Pattern
Recognition, 2008.
[12] H. Chen, C. Shen, and P. Tsai, “Edge-based automatic white balancing with linear illuminantconstraint,” in Visual Communications and Image
Processing, 2007.
[13] A. Gijsenij, T. Gevers, and J. van de Weijer, “Generalized gamut mapping using image derivative
structures for color constancy,” IJCV, vol. 2–3, pp.
127–139, 2010.
[14] J. van de Weijer, T. Gevers, and A. Gijsenij,“Edge-based color constancy,” IEEE TIP, vol. 16,
no. 9, pp. 2207–2214, 2007.
[15] K. Barnard, V. Cardei, and B. Funt, “A comparison of computational color constancy algorithms;
part i,” IEEE TIP, vol. 11, no. 9, pp. 972–984, 2002.
[16] K. Barnard and B. Funt, “Color constancy with
specular and non-specular surfaces,” in Proc. CIC,
1999, pp. 114–119.
[17] R. Tan, K. Nishino, and K. Ikeuchi, “Color
constancy through inverse-intensity chromaticity
space,” J. Opt. Soc. Am. A, vol. 21, no. 3, pp. 321–
334, 2004.
[18] K. Barnard, G. Finlayson, and B. Funt, “Color
constancy for scenes with varying illumination,”

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. XX, NO. Y, DATEComp. Vis. Im. Und., vol. 65, no. 2, pp. 311–321,
1997.
[19] G. Finlayson, B. Funt, and K. Barnard, “Color
constancy under varying illumination,” in Proc.
ICCV, 1995, pp. 720–725.
[20] G. Finlayson, S. Hordley, C. Lu, and M. Drew,“On the removal of shadows from images,” IEEE
TPAMI, vol. 28, no. 1, pp. 59–68, 2006.
[21] J. Geusebroek, R. van den Boomgaard, A. Smeulders, and H. Geerts, “Color invariance,” IEEE
TPAMI, vol. 23, no. 12, pp. 1338–1350, 2001.
[22] T. Gevers and H. Stokman, “Classifying color
edges in video into shadow-geometry, highlight, or material transitions,” IEEE T. Multimedia,
vol. 5, pp. 237–243, 2003.
[23] J. van de Weijer, T. Gevers, and J. Geusebroek, “Edge and corner detection by photometric quasi-invariants,” IEEE TPAMI, vol. 27, no. 4,
pp. 625–630, 2005.
[24] J. von Kries, “Influence of adaptation on theeffects produced by luminous stimuli,” in Sourcesof Color Vision, D. MacAdam, Ed. MIT Press,
1970, pp. 109–119.
[25] W. Freeman and E. Adelson, “The design and useof steerable filters,” IEEE TPAMI, vol. 13, no. 9,
pp. 891–906, 1991.
[26] A. Gijsenij and T. Gevers, “Shadow edge detection using geometric and photometric features,”in IEEE International Conference on Image Processing, 2009, pp. 1–8.
[27] J. Zhu, K. Samuel, S. Masood, and M. Tappen,“Learning to recognize shadows in monochromatic natural images,” in Proc. of the IEEE Conf.
on Computer Vision and Pattern Recognition, 2010,
pp. 223–230.
[28] S. Hordley and G. Finlayson, “Reevaluation ofcolor constancy algorithm performance,” J. Opt.
Soc. Am. A, vol. 23, no. 5, pp. 1008–1020, 2006.
[29] K. Barnard, L. Martin, B. Funt, and A. Coath, “Adata set for color research,” CRA, vol. 27, no. 3,
pp. 147–151, 2002.
[30] F. Ciurea and B. Funt, “A large image databasefor color constancy research,” in Proc. CIC, 2003,
pp. 160–164.
[31] S. Bianco, G. Ciocca, C. Cusano, and R. Schettini, “Improving color constancy using indooroutdoor image classification,” IEEE TIP, vol. 17,
no. 12, pp. 2381–2392, 2008.
[32] P. Gehler, C. Rother, A. Blake, T. Minka, andT. Sharp, “Bayesian color constancy revisited,”in Proc. of the IEEE Conf. on Computer Vision and
Pattern Recognition, 2008, pp. 1–8.
[33] A. Gijsenij, T. Gevers, and M. Lucassen, “A perceptual analysis of distance measures for color
constancy,” J. Opt. Soc. Am. A, vol. 26, no. 10, pp.
2243–2256, 2009.

14

Arjan Gijsenij is currently scientific software
consultant for Alten PTS. Formerly, he was
postdoctoral researcher at the University of
Amsterdam. He received his M.Sc. degreein Computer Science at the University ofGroningen in 2005, and obtained his Ph.D.
degree at the University of Amsterdam in2010. His main research interests are in thefield of color in computer vision and psychophysics. He has published papers in highimpact journals and conferences, and servedon the program committee of several conferences and workshops.

Theo Gevers is an Associate Professorof Computer Science at the University of
Amsterdam, The Netherlands, where he isalso teaching director of the M.Sc. of Artificial Intelligence. He currently holds a VICIaward (for excellent researchers) from theDutch Organisation for Scientific Research.
His main research interests are in the fundamentals of contentbased image retrieval,color image processing and computer vision
specifically in the theoretical foundation ofgeometric and photometric invariants. He is chair of various conferences and he is associate editor for the IEEE Transactions on Image
Processing. Further, he is program committee member of a numberof conferences, and an invited speaker at major conferences. He is alecturer of post-doctoral courses given at various major conferences(CVPR, ICPR, SPIE, CGIV). He is member of the IEEE.

Joost van de Weijer received his MSc (Delft
University of Technology) in 1998 and his
PhD (University of Amsterdam) in 2005. He
was Marie Curie Intra-European fellow in INRIA Rhone-Alpes. He was awarded the Ramon y Cajal Research Fellowships in Computer Science by the Spanish Ministry of
Science and Technology. He is based in theComputer Vision Center in Barcelona (Spain)
since 2008. He is member of the IEEE.