3D shape perception from combined depth cues in human visual cortex
Authors: Heinrich H B√ºlthoff
Arne Deubelius
Zoe Kourtzi
Andrew E Welchman
Verena Conrad
Journal: Nature Neuroscience (2005)
Discipline: NEUROSCIENCE & BEHAVIOR
Keywords: 3D structure perception; neural architecture; visual information; depth cues; fMRI responses
Affiliations: Max-Planck Institute for Biological Cybernetics, Postfach 2169, 72012 T√ºbingen, Germany.

Abstract:  Our perception of the world's three-dimensional (3D) structure is critical for object recognition, navigation and planning actions. To accomplish this, the brain combines different types of visual information about depth structure, but at present, the neural architecture mediating this combination remains largely unknown. Here, we report neuroimaging correlates of human 3D shape perception from the combination of two depth cues. We measured fMRI responses while observers judged the 3D structure of two sequentially presented images of slanted planes defined by binocular disparity and perspective. We compared the behavioral and fMRI responses evoked by changes in one or both of the depth cues. fMRI responses in extrastriate areas (hMT+/V5 and lateral occipital complex), rather than responses in early retinotopic areas, reflected differences in perceived 3D shape, suggesting 'combined-cue' representations in higher visual areas. These findings provide insight into the neural circuits engaged when the human brain combines different information sources for unified 3D visual perception. 

Retracted: No

PubPeer comments:
 u201cVisual environments are defined by multiple cues to depth structure, such as binocular disparity, perspective, texture, shading and motion. The computations and neural mechanisms required to extract this information from the retinal images are likely to be radically different, but despite this, we perceive coherent 3D structures; somehow, the information provided by different depth cues is combined by the brain. (Landy, M.S., Maloney, L.T., Johnston, E.B. Young, M. Measurement and modeling of depth cue combination u2013 in defense of weak fusion.Vision Res.35, 389u2013412 (1995)). To rephrase more simply: We perceive coherent 3D structures by combining u201ccues to depth structureu201d such as binocular disparity, perspective, texture, shading and motion. This claim has no basis and no substance. The single reference provided is oddly irrelevant u2013 can such a broad and weighty statement be supported by a (then) 15-year-old paper merely framed as a u201cdefenseu201d of a position that was clearly not even conventional? What are readers to make of it? This is a detail. The main point is that none of the u201ccuesu201d listed, individually or collectively, can explain shape perception, which is primarily a matter of perceptual organization. 1 None of the listed u201ccuesu201d is necessary to explain shape perception. This is self-evident. We may use line-drawings to produce any, or almost any, coherent 3D shape we like. With no u201cperspectiveu201d (which I assume refers to converging lines), no motion, no texture, no shading, no binocular disparity. I presume that this is achieved via brain mechanisms, and that these mechanisms are in play even in the presence of the u201ccues.u201d Any investigation of the subject that neglects to consider this factor (whatever form it takes in the brain) is neglecting a major confound. 2 To call u201cbinocular disparity, perspective, texture, shading and motionu201d cues to shape perception is in large part to reverse cause and effect. u201cPerspective,u201d referring to convergent lines, isn t a u201ccueu201d because converging lines do not necessarily produce the impression of depth u2013 it depends on the whole structure structures involved. A drawing of an ordinary parallelipiped falsifies this feature as a u201ccueu201d to depth; When the figure is extended along the parallel (in 2D) diagonals, the lines may appear to diverge, rather than converge, in depth. This may be seen, for example, in Figure 3b of Yoonessi et al (2008) https: journals.sagepub.com doi abs 10.1068 p6282?journalCode=peca . 3 u201cTextureu201d as cue typically refers to the fact that elements of a homogeneous pattern that is slanted relative to the projection plane will shrink with distance. This is simply a special case of the fact that the projection of any surface shrinks with distance from the projection plane. In order to act as a shape cue, any such texture must be bounded, e.g. by a line or by a different pattern or a homogeneous surface (e.g. a white page). The grouping of elements into a textured unit precedes the application of a texture cue, since isolated elements can t constitute a texture. Furthermore, the form of this outline is far more consequential than any texture pattern within it. For example, if we added a dot pattern to a side of our parallelipiped with larger dots nearer the u201cbacku201d end growing smaller and smaller toward the u201cfront,u201d the shape will not change. But again, the factor of form of the projection is not considered. 4 The invocation of shading as a cue to shape or depth rests on the assumption that u201cdarker is deeper,u201d to quote Tyler and Chen (2015). This is obviously invalid, since clearly we don t always interpret a darker surface as being further away u2013 on a checkerboard, for example. Or to take our parallelipiped again as an example, we could paint the front face black and it wouldn t suddenly recede perceptually. Even if the darkening is due to reduced illumination, we could be dealing with shadows not shading. The dispositive factor, again, is figural u2013 the figural organization of the stimulation. We don t detect shading and infer shape u2013 we infer shape and that determines whether darker is perceived as shadow, shading, albedo. 5 u201cMotionu201d is not a cue because the perception of motion is contingent on figural organization. The Mach card demo is a good example. In one of the two perceived organizations, it appears to move, in order to maintain a stable shape percept. It is not the motion that organizes the perceived depth relations, but the figural constraints the organize the depth relations that create the perceived motion. Actual motion is never a requirement for perceived motion u2013 movies are an obvious example of that. Moreover, there is a famous demonstration by Karl Duncker showing that moving a shape framing another (in this case a rectangle surrounding a dot) will cause the enclosed figure to appear to move in the opposite direction (while the frame appears still). A familiar example of this effect is the experience of our car sliding backwards when the large truck next to us starts moving forward. In this well-cited study, the u201ccue combinationu201d theory of shape perception has been taken as an a priori premise, and data interpreted in this light. Figural organization, the dispositive u201ccue,u201d is never considered. I think this is a problem. ,