Seeing slow and seeing fast : Two limits on perception

Video cameras have a single temporal limit set by the frame rate. The human visual system has multiple temporal limits set by its various constituent mechanisms. These limits appear to form two groups. A fast group comprises specialized mechanisms for extracting perceptual qualities such as motion direction, depth, and edges. The second group, with coarse temporal resolution, includes judgments of the pairing of color and motion, the joint identification of arbitrary spatially separated features, the recognition of words, and high-level motion. These temporally coarse percepts may all be mediated by high-level processes. Working at very different timescales, the two groups of mechanisms collaborate to create our unified visual experience. Spatial versus temporal scales Our visual system has a spatial grain, making small details completely invisible, such as the individual molecules forming the surface you are currently viewing. Less appreciated is that our processing of time is also confined to coarse scales. Events confined to thousands of a second, and sometimes even hundredths of a second or longer, are not perceived. For example, until brief-exposure photography was invented in the 1870s, it was not known whether a galloping horse lifts all four hooves off the ground at any one time [1]. Just as the microscope probed small spatial scales long before stroboscopic photography investigated small temporal scales, so vision scientists have traditionally concentrated on visionʼs spatial resolution rather than its temporal resolution. But recent years have seen an upsurge in interest in temporal processing. Much of this growth has been in areas of temporal illusions, such as distortions in time perception [2], asynchronies in binding the features of an object [3], and possible interactions between object motion and the time an object is perceived [4]. Recent reviews of these topics are available [5-7], but missing from the literature is a modern synopsis of the temporal limits of vision—the timescales on which the machinery of perception operates. These temporal limits set the foundation necessary to understand temporal illusions, and in bringing together the diverse psychophysical literature on temporal limits, a broad generalization emerges regarding the functional organization of the visual system. Understanding multiple temporal scales Letʼs begin with a naïve question—does your visual experience display the world at five, twenty, or one hundred frames per second? In the case of the cinema, we know there is a frame rate, a temporal grain of twenty-four per second. But our visual system does not operate like movie equipment. Visual processing is more continuous [8]—although see [9] for the suggestion of a discrete component. In continuous processing, although there is no frame rate because discrete snapshots are not taken, the system still can be said to have a temporal grain. The grain, or temporal interval over which the system blurs information together, is known as temporal resolution. A movie camera has one temporal resolution, reflecting its frame rate. But the visual system has multiple temporal resolutions. The high temporal resolution of some of our perceptual mechanisms presents a problem for the cinematographer working with a film shot, as most are, at twentyfour pictures per second. If films were projected at 24 frames per second, the flicker would be obvious and annoying, because our flicker perception mechanism is fast enough to detect 24-Hz flicker. To thwart both flicker perception and some motion artifacts that our fast motion mechanisms are sensitive to, cinema projectors flick each frame on and off twice before moving on to the next frame. The resulting 48 per second (48 Hz) flicker rate is almost invisible to us. Although the technique eliminates the flicker perception problem, one might expect other problems to remain. As each object is shown twice in the same location before moving on, a cinematic car chase might appear as a stuttering procession rather than as the familiar fluid and seamless experience. Fortunately for the cinematographer, our visual perception fails to register the stuttering—a consequence of the coarse temporal scale at which most perceptual mechanisms process the scene. As will be described below, the 24 pictures per second rate is already too fast for many of our visual mechanisms. Our perception comprises some mechanisms, like flicker and motion, that inform us of rapid changes, while others are restricted to much slower changes and thus fooled by a 24 Hz simulacrum of reality. Two of the temporal limits underpinning experience can be demonstrated with a simple animation. In Movie 1, adjacent striped patches alternate between leftward-tilted and rightward-tilted. When the alternation rate is only a few per second (top row), it is easy to perceive whether the two patches are always opposite in orientation (top left) or are always oriented in the same way (top right). If the patches are alternated at a faster rate, say between 8 and 12 frames per second (second row), it becomes difficult or impossible to judge whether the patches have the same orientation or opposite orientations [10]. Interestingly, although too fast for the temporal relationship of the two features to be perceived, this rate is nevertheless slow enough to identify the two orientations [11] and to detect the flicker. Not until the rate is increased much further would the movie appear completely fused (bottom row), with neither the relationship of the two patches, their individual patterns, nor their flicker perceived. Each speed limit likely corresponds to the narrowest temporal scale at which a perceptual mechanism operates. Consider the perception of fluorescent lights, which illuminate the workdays of most office workers. Fluorescent lights flicker— 120 times per second (120 Hz) in the US and 100 Hz in many other places. We do not perceive this flicker because our underlying visual detectors average their inputs over a longer interval. Indeed, our visual mechanisms average over 20 milliseconds or more, blurring together an entire 10-ms cycle of 100 Hz flicker and therefore obliterating the changes in luminance in the fluorescent lights and in the 48-Hz flicker of the double-projected cinema. At first, one might think this integration interval could be revealed by presenting a stimulus a single time rather than in a cycle, and determining the shortest duration for which it can still be perceived. But a 5-ms stimulus flash, if sufficiently high contrast, can be analyzed even by a mechanism that operates at a slow 100-ms scale, because averaging over 100 ms will not eliminate the stimulus information. This point has been key to a debate in the literature on the timescale of visual grouping [12]. The alternating displays referred to in the present article are specifically designed to provide no information when a mechanism integrates over the cycle of alternation. Restricting ourselves to such displays will lead us to ignore for now (but see Box 1) the interesting literature on the effect of varying presentation duration of a single stimulus. Although many studies have probed the temporal limits of visual judgments using the alternating stimulus technique, because these studies have used very different stimuli and tasks, the results usually cannot be compared with quantitative precision. However, in some cases the speed limits found differ so much that the differences are unlikely to be caused by experiment details. Figure 1 shows some of these limits. The limits fall into two clusters, slow and fast, separated by a large gap. Fast limits on visual percepts The fast cluster includes flicker perception. The “flicker fusion” limit, like others, depends on mean luminance, luminance contrast, and other factors, but is usually at least 50 Hz on ordinary computer screens. Only six other types of visual judgments have been found to be of comparable speed: • first-order motion: A moving sinusoidal grating with bright peaks and dark troughs is seen through a window. Even when its velocity is so high that 30 of its bright peaks pass each location per second (30 Hz), observers still perceive its direction of motion [13]. If the input to motion detectors was blurred or jumbled over this timescale, determining the motion direction would be impossible. Flicker is a degenerate case of motion and the same bank of fast detectors could be responsible for both the motion and flicker limits. • depth from binocular disparity: Two identical gratings, one in each eye, are viewed through identical windows. One is spatially shifted relative to the other, introducing a binocular disparity that is perceived as depth. Even when they move at a 30-Hz rate, the depth is still perceived [14]. If the input to depth mechanisms were blurred at a >30 ms timescale, detecting the depth would be impossible at this rate. • edges and texture boundaries: Against a gray background, a field of white dots is adjacent to a field of black dots. Both fields are set in rapid alternation between white and black, but out of phase—when one is white, the other is black. The conspicuous boundary between them is perceived even at fast rates of 30 Hz [15, 16], while at faster rates the black and white are averaged by the brain into the same color as the background. Texture boundaries defined by orientation differences can also be seen at fast rates [11]. • binding of color and orientation sharing a spatial location: If a red, righttilted patch is alternated with a green, left-tilted patch, it is easy to distinguish from the complementary pairing of red left alternating with green right, even at fast 20 Hz rates [17] (demo at www.psych.usyd.edu.au/staff/alexh/research/binding/ ) • color and orientation as individual features: Fast binding of color and orientatio