Mental geometry of three-dimensional size perception
Authors: Qasim Zaidi
Akihito Maruya
Journal: Journal of Vision (2020)
Discipline: CLINICAL MEDICINE
Keywords: poses; sizes; shapes; objects; projective geometry
Affiliations: Graduate Center for Vision Research, State University of New York, New York, NY, 

Abstract:  Judging the poses, sizes, and shapes of objects accurately is necessary for organisms and machines to operate successfully in the world. Retinal images of three-dimensional objects are mapped by the rules of projective geometry and preserve the invariants of that geometry. Since Plato, it has been debated whether geometry is innate to the human brain, and Poincare and Einstein thought it worth examining whether formal geometry arises from experience with the world. We examine if humans have learned to exploit projective geometry to estimate sizes and aspects of three-dimensional shape that are related to relative lengths and aspect ratios. Numerous studies have examined size invariance as a function of physical distance, which changes scale on the retina. However, it is surprising that possible constancy or inconstancy of relative size seems not to have been investigated for object pose, which changes retinal image size differently along different axes. We show systematic underestimation of length for extents pointing toward or away from the observer, both for static objects and dynamically rotating objects. Observers do correct for projected shortening according to the optimal back-transform, obtained by inverting the projection function, but the correction is inadequate by a multiplicative factor. The clue is provided by the greater underestimation for longer objects, and the observation that they seem to be more slanted toward the observer. Adding a multiplicative factor for perceived slant in the back-transform model provides good fits to the corrections used by observers. We quantify the slant illusion with two different slant matching measurements, and use a dynamic demonstration to show that the slant illusion perceptually dominates length nonrigidity. In biological and mechanical objects, distortions of shape are manifold, and changes in aspect ratio and relative limb sizes are functionally important. Our model shows that observers try to retain invariance of these aspects of shape to three-dimensional rotation by correcting retinal image distortions due to perspective projection, but the corrections can fall short. We discuss how these results imply that humans have internalized particular aspects of projective geometry through evolution or learning, and if humans assume that images are preserving the continuity, collinearity, and convergence invariances of projective geometry, that would simply explain why illusions such as Ames' chair appear cohesive despite being a projection of disjointed elements, and thus supplement the generic viewpoint assumption. 

Retracted: No

PubPeer comments:
 The quantitative claims being made by Maruya Zaidi are being framed as general claims when they clearly are not. The degree of perceptual correction for foreshortening in the projected image of a 3D object is contingent on the shape of the projection (as well as on its orientation in the 2D plane). This is the reason that perfectly flat, fronto-parallel surfaces (pictures) may be made to appear 3D. Some shapes in pictures are perceptually treated as though they were projections from slanted objects in the physical world. Other shapes are not, regardless of whether they could, in fact, be projections of slanted surfaces. For example, a 3D surface shaped like a trapezoid and rotated so that its projection is rectangular will be perceived as a fronto-parallel rectangle and will *not* be corrected for *actual* foreshortening due to actual slant. The conditions in Zaidi s experiment are pictorial (monocular), so the same rules apply. In addition to projection shape, the role of projection *orientation* in the degree of u201cundoingu201d of putative foreshortening is the basis of the well-known Shepard Illusion, including a [parallelogram-only version] . I ve also explored the radical changes in the perception of 3D line drawings that follow changes in planar orientation and their implications ([Maniatis 2010] ; [Maniatis 2017] ). What all of this means is that statements like those quoted below may apply to the particular conditions of this study, but, again, can t be treated as general: u201cWe show systematic underestimation of length for extents pointing toward or away from the observer, both for static objects and dynamically rotating objects.u201d u201cObservers do correct for projected shortening according to the optimal back-transform, obtained by inverting the projection function, but the correction is inadequate by a multiplicative factor.u201d Again, it all depends on the shape of the projection. It also seems as though the authors are making a false dichotomy between shape constancy and what they re describing as size constancy: u201cHowever, it is surprising that possible constancy or inconstancy of relative size seems not to have been investigated for object pose, which changes retinal image size differently along different axes.u201d Shape constancy is equivalent to correction for (putative) projective distortion, including both change in angles and changes in relative foreshortening. There is no such thing as a u201cposelessu201d object. The Shepard illusion is of course a demonstration of the fact that perceptually inferred u201cposeu201d produces *different* corrections for putative foreshortening. A methodological issue: The stimuli used here are pictorial, on a computer screen. Observers were asked to estimate lengths of apparently slanted extents. When we isolate a part of an image, the contextual effects contingent on the whole image may be reduced or lost. For example, the angles in a Necker cube appear like all right angles, but if we narrow our focus we can see the angles as acute or obtuse (or right) as the case may be. So a reduction in perceived length relative to the u201crealityu201d as framed by the investigators is to be expected when we narrow our focus. Artists need to do this to make veridical drawings. , 
  We discuss how these results imply that humans have internalized particular aspects of projective geometry through evolution or learning and if humans assume that images are preserving the continuity, collinearity, and convergence invariances of projective geometry that would simply explain why illusions such as Ames chair appear cohesive despite being a projection of disjointed elements, and thus supplement the generic viewpoint assumption. The notion that the retinal image is interpreted based on  internalized aspects of projective geometry and the preservation of  invariants doesn t make sense; it s a kind of illusion. The reason is simple: Even if we ignore the (non-trivial) problem of figure-ground segregation, and assume that the retinal projections from individual objects are sent pre-packaged to the visual system to do its thing, it is still the case that every retinal projection can potentially have come from an infinite number of 3D objects. If there are invariants connecting 3D shape and projection, such invariants would apply to *each* possible underlying 3D shape. Putative invariants could only be useful, in other words, if we *already knew* what object had made the projection. And if we knew that, the job would be done. But we don t. Strange as it may seem, the problem of converting the retinal projection into a 3D perceptual representation must be resolved without invoking projective geometry (except in retrospect). ,