2004 Special Issue Hue geometry and horizontal connections

AbstractPrimate visual systems support an elaborate specialization for processing color information. Concentrating on the hue component, weobserve that, contrary to Mondrian-like assumptions, hue varies in a smooth manner for ecologically important natural imagery. To representthese smooth variations, and to support those information processing tasks that utilize hue, a piecewise smooth hue ﬁeld is postulated. Thegeometry of hue-patch interactions is developed analogously to orientation-patch interactions in texture. The result is a model for long-range(horizontal) interactions in the color domain, the power of which is demonstrated on a number of examples. Implications for computer imageprocessing, computer vision, visual neurophysiology and psychophysics are discussed.q 2004 Elsevier Ltd. All rights reserved. Keywords: Hue curvature; Horizontal connection; Cytochrome oxidase blobs 1. IntroductionThe visual world is a rich source of color-codedinformation. Ripe red fruit standing out from a backgroundof green leaves immediately suggests one advantage ofcolor selectivity to frugivore primates (Sumner & Mollon,2000a,b), and the elaborate physiological specialization forprocessing color attests to its use. Photoreceptors evolveddifferential spectral responses, and cytochrome oxidasestaining has revealed anatomical blobs in cortex. Neuronswith opponent receptive ﬁeld structure provide maps.But there is much more to the cortical processing ofcolor: neural selectivities differ with context (Wachtler,Sejnowski, & Albright, 2003), and ﬁlling-in phenomena(Redies & Spillmann, 1981) suggest boundary and surfacematerial interactions. The interaction between surfacegeometry, reﬂectivity, and light sources must matter(Beck, 1972; Horn, 1986a); at some level, color-constantbehaviors emerge (Zeki, 1993) in a manner such that, asHelmholtz famously observed, the illuminant is discounted.Artists have intuitively exploited this (Albers, 1987), andexperience in computer graphics indicates how critical itcan be. All of the above, taken together, suggest a muchbroader and deeper role for color processing in the inferenceof scene structure than the simple detection of ripe fruit, andour goal in this paper is to sketch one part of a this broaderview of color processing.We concentrate almost entirely on hue, and illustrate, inSection 2, different putative roles for it. Although hueplayed a fascinating role in the discovery of color opponentprocessing, it is surprisingly understudied in computervision (Forsyth & Ponce, 2002). But the time is right toconsider hue more directly, because neurophysiologicalevidence of hue maps is now emerging (Hanazawa,Komatsu, & Murakami, 2000; Wachtler et al., 2003; Xiao,Wang, & Felleman, 2003). In particular, it is unlikely thattrichromaticity would have re-evolved without providingbasic information processing advantages. We argue, inparticular, that when the full variability of fruit imagery isconsidered, the detection problem becomes rather moresubtle. This serves, in turn, as a bridge to more abstractquestions about the inferences of scene structure.Our paper is mainly theoretical, and relies on an analogybetween two physiological specializations in visual cortex.While differences between the cytochrome oxidase blobsand the interblobs in visual cortex are normally stressed, weshall take the opposite tack and stress a similarity. Oneprimary biophysical difference—metabolic capability—is