Image statistics and the perception of surface qualities
Authors: Lavanya Sharan
Edward H Adelson
Isamu Motoyoshi
Shin&#x27;ya Nishida
Journal: Nature (2007)
Discipline: Multidisciplinary
Keywords: surfaces; appearance; gloss; reflectance; skewness
Affiliations: Human and Information Science Lab, NTT Communication Science Labs, Nippon Telegraph and Telephone Corporation, 3-1 Morinosato-Wakamiya, Atsugi, 243-0198, Japan. 

Abstract:  The world is full of surfaces, and by looking at them we can judge their material qualities. Properties such as colour or glossiness can help us decide whether a pancake is cooked, or a patch of pavement is icy. Most studies of surface appearance have emphasized textureless matte surfaces, but real-world surfaces, which may have gloss and complex mesostructure, are now receiving increased attention. Their appearance results from a complex interplay of illumination, reflectance and surface geometry, which are difficult to tease apart given an image. If there were simple image statistics that were diagnostic of surface properties it would be sensible to use them. Here we show that the skewness of the luminance histogram and the skewness of sub-band filter outputs are correlated with surface gloss and inversely correlated with surface albedo (diffuse reflectance). We find evidence that human observers use skewness, or a similar measure of histogram asymmetry, in making judgements about surfaces. When the image of a surface has positively skewed statistics, it tends to appear darker and glossier than a similar surface with lower skewness, and this is true whether the skewness is inherent to the original image or is introduced by digital manipulation. We also find a visual after-effect based on skewness: adaptation to patterns with skewed statistics can alter the apparent lightness and glossiness of surfaces that are subsequently viewed. We suggest that there are neural mechanisms sensitive to skewed statistics, and that their outputs can be used in estimating surface properties. 

Retracted: No

PubPeer comments:
  We find evidence that human observers use skewness, or a similar measure of histogram asymmetry, in making judgements about surfaces. When the image of a surface has positively skewed statistics, it tends to appear darker and glossier than a similar surface with lower skewness.u201d u201cPerceived lightness and glossiness may be based on the skewnessof the luminance histogramsu201d What could be easier to refute than the claims of this paper, which happen to be fundamentally at odds with Adelson (1993)? In that famous paper with its u201ccorrugated plaidu201d illustration, Adelson states: u201cBrightness judgments cannot be simply explained with low-level mechanisms. Geometrical changes that should be inconsequential for low-level mechanisms can cause dramatic changes in the brightness report. In contrast, Motoyoshi et al give pride of place to geometry-blind luminance histograms, linked to u201cearly stageu201d visual neurons. It s no surprise that an image that contains a relatively larger number of low-luminance points might appear darker than a similar one with more high-luminance points. The main claim here pertains to the perception of gloss (or lightness, as the above quote shows), and the idea that this feature of a percept is functionally connected to its (structure-blind) luminance histogram. The authors come to this conclusion on the basis of a few images they have selected on unknown criteria. The danger of sampling error is very real. As far as corresponding to the general case, the histogram proferred as conducive to the perception of gloss is only one of many possible histograms we could extract from gloss-producing images. I don t think the paper s u201cskewed positiveu201d histogram would apply to the images below (see bottom of page). It should be clear that by varying the surfaces that the fork, for example, is reflecting, we could alter the fork s luminance histograms without undermining the perception of the fork as a coherent, shiny surface, as it should also be clear that the shape of a surface u2013 whether it s smooth or bumpy, whether the bumps are shallow or deep, and how it is illuminated - for example u2013 will affect these histograms. Moreover, shadows, highlights, etc, have locations and shapes. A deconstructed collection of points varying in intensity, and their frequencies, has nothing to tell us about these very relevant figural relationships. The illustrations in Adelson (1993) were designed to show that changes in structure could radically affect the perception of brightness of surfaces even if luminances are left unchanged. Here, for some reason, these truths go out the window: u201cWe do not know what aspects of image structure determine u2018surfacenessu2019 or u2018highlightsu2019.u201d Clearly, Adelson (1993) knew enough to create and manipulate perceived surfaces and their perceived brightness (u201cmechanisms, involving contours, junctions, and grouping, appear to be critical in explaining many lightness phenomenau201d). There are principles underlying this practical knowledge, and they re not structure-blind-histogram-based. ![file] ![file] ![file] ,