Spatiotemporal unit formation

Findings in dynamic unit formation suggest that completion processes reflect the optics of our world. Dynamic unit formation may depend on patterns of motion signals that are consistent with the causes of optical changes. In addition, dynamic completion conforms to a local curvature minimization constraint. Such relational aspects of vision are important to consider in linking perceptual experience and neural activity. Much of the discussion of filling in has focused on static displays. Perceptual unit formation is also evident in moving displays, and research on surface, boundary, and path completion of moving objects highlights the importance of ecological considerations in understanding the link between neural activity and perceptual experience. At the personal level both modal and amodal completion of objects is observed. A substantial portion of Michotte’s original discussion of amodal perception included dynamic displays (Michotte et al. 1964). For example, in “tunneling,” an object will be seen as moving along a continuous path behind an occluder when the visible parts of the moving object’s path align spatially and temporally. Apparent motion, where an object appears to move back and forth between two locations, is an example of modal completion across time. It might be tempting to hypothesize that neurons that respond to a moving point (and apparent motion) are the locus for the neural completion of an object’s changing location; however, a consideration of the optics of our world suggests that the link between neural and personal is not so simple. Sequential luminance changes on the retina may arise from two different environmental events, the motion of an object or the sequential occlusion of multiple objects or textural elements. In the first case, the local motion signals are related directly to the motion in the world; the motion signals that occur as things appear and disappear, however, are not. Hence the latter motion signals should not serve as representations of object locations in the world. These signals are not random; their spatiotemporal pattern is lawfully related to an aspect of the occluding object – the orientation of the occluding edge (Shipley & Kellman 1994; 1997). Each pair of appearances or disappearances produces a motion signal whose direction and magnitude is a function of the orientation of the occluding edge and the relative position of the changing elements. As a consequence, any pair of motion signals could be used to recover the orientation of the occluding edge. Specifically, if two motion signals, represented by vectors, are positioned so that they have a common origin, the tips of the vectors define the local orientation of the occluding edge. Discriminating between the patterns of motion signals that accompany motion and dynamic occlusion allows the visual system to use the motion signals to identify boundaries of surfaces that are not specified by luminance, texture, or other static differences. Such boundary formation processes could aid in the identification of objects seen while moving through a cluttered world. The distinction between local element motion and occlusion has been reported by a number of researchers who have found that one of two percepts may be experienced in displays where elements appear and disappear. Depending on stimulus conditions, such as the temporal interval between changes, observers may report either (1) an edge hiding and revealing elements or (2) motion of individual elements (Shipley & Kellman 1994; Sigman & Rock 1974; Wallach 1935/1996). A parallel dichotomy is observed in static displays where a tangent discontinuity is perceived as either a corner in the world or a consequence of partial occlusion. There are a number of other parallels between static illusory contours and spatiotemporally defined edges. In both cases the bounded region appears to have a surface quality that differs from the surround (Cicerone et al. 1995; Cunningham et al. 1998), but a surface quality difference is not necessary to see a bounding edge (Kellman & Loukides 1987; Shipley & Kellman 1994). In addition, the filled in edge is almost always seen as smooth. Corners in illusory contours are rarely reported and when shown a spatiotemporally defined hexagon subjects frequently mistake it for a circle (Shipley & Kellman 1994). Minimization of change, both across surfaces and along edges, would appear to be a general property of filling in. The observation of covariation in perceptual aspects of an event, as well as constraints on how filling in occurs, are relevant to issues of representations in vision. Conceiving of perceptual representations as encoding all aspects of a scene is indeed problematic. A more limited meaning of representation may be appropriate, and may help focus on the importance of relationships. Gallistel (1990) offers a definition of neural representations as events or states of the nervous system that are isomorphic to events or states of the world. This definition does not require all aspects of the world be encoded, only the ones that are critical for action. Unlike many uses of the term representation, this is purely mathematical, so it focuses on what relationships (or functions) present in the represented system are preserved in the representing system (e.g., when numbers are used to represent quantities of objects, some relations such as addition and division are preserved, whereas others, such as whether the objects are soft or hard, are not). In building a bridge between the personal and neural, it may be useful to try to understand which mathematical relationships apply in the representing system. Considering both how the nervous system may instantiate computations (Gallistel [1990] reviews a number of cases where the nervous system appears to use vector representations and vector addition occurs as a consequence of the spatial structure of the nervous system) and the ecological relationships, which suggest that certain perceptual experiences will be coupled (Epstein [1982] provides a review of cases where two perceptual experiences are clearly linked) should be helpful.