Decoding of MSTd Population Activity Accounts for Variations in the Precision of Heading Perception

Humans and monkeys use both vestibular and visual motion (optic flow) cues to discriminate their direction of self-motion during navigation. A striking property of heading perception from optic flow is that discrimination is most precise when subjects judge small variations in heading around straight ahead, whereas thresholds rise precipitously when subjects judge heading around an eccentric reference. We show that vestibular heading discrimination thresholds in both humans and macaques also show a consistent, but modest, dependence on reference direction. We used computational methods (Fisher information, maximum likelihood estimation, and population vector decoding) to show that population activity in area MSTd predicts the dependence of heading thresholds on reference eccentricity. This dependence arises because the tuning functions for most neurons have a steep slope for directions near straight forward. Our findings support the notion that population activity in extrastriate cortex limits the precision of both visual and vestibular heading perception.

Related papers:

title: The Effect of Correlated Variability on the Accuracy of a Population Code
abstract: We study the impact of correlated neuronal firing rate variability on the accuracy with which an encoded quantity can be extracted from a population of neurons. Contrary to widespread belief, correlations in the variabilities of neuronal firing rates do not, in general, limit the increase in coding accuracy provided by using large populations of encoding neurons. Furthermore, in some cases, but not all, correlations improve the accuracy of a population code.

title: Neural correlations, population coding and computation
abstract: How the brain encodes information in population activity, and how it combines and manipulates that activity as it carries out computations, are questions that lie at the heart of systems neuroscience. During the past decade, with the advent of multi-electrode recording and improved theoretical models, these questions have begun to yield answers. However, a complete understanding of neuronal variability, and, in particular, how it affects population codes, is missing. This is because variability in the brain is typically correlated, and although the exact effects of these correlations are not known, it is known that they can be large. Here, we review studies that address the interaction between neuronal noise and population codes, and discuss their implications for population coding in general.

title: Bayesian inference with probabilistic population codes
abstract: Recent psychophysical experiments indicate that humans perform near-optimal Bayesian inference in a wide variety of tasks, ranging from cue integration to decision making to motor control. This implies that neurons both represent probability distributions and combine those distributions according to a close approximation to Bayes' rule. At first sight, it would seem that the high variability in the responses of cortical neurons would make it difficult to implement such optimal statistical inference in cortical circuits. We argue that, in fact, this variability implies that populations of neurons automatically represent probability distributions over the stimulus, a type of code we call probabilistic population codes. Moreover, we demonstrate that the Poisson-like variability observed in cortex reduces a broad class of Bayesian inference to simple linear combinations of populations of neural activity. These results hold for arbitrary probability distributions over the stimulus, for tuning curves of arbitrary shape and for realistic neuronal variability.

title: Statistically Efficient Estimation Using Population Coding
abstract: Coarse codes are widely used throughout the brain to encode sensory and motor variables. Methods designed to interpret these codes, such as population vector analysis, are either inefficient (the variance of the estimate is much larger than the smallest possible variance) or biologically implausible, like maximum likelihood. Moreover, these methods attempt to compute a scalar or vector estimate of the encoded variable. Neurons are faced with a similar estimation problem. They must read out the responses of the presynaptic neurons, but, by contrast, they typically encode the variable with a further population code rather than as a scalar. We show how a nonlinear recurrent network can be used to perform estimation in a near-optimal way while keeping the estimate in a coarse code format. This work suggests that lateral connections in the cortex may be involved in cleaning up uncorrelated noise among neurons representing similar variables.

title: Probability density estimation for the interpretation of neural population codes.
abstract: 1. Electrophysiological recording data from multiple cells in motor cortex and elsewhere often are interpreted using the population vector method pioneered by Georgopoulos and coworkers. This paper proposes an alternative method for interpreting coding across populations of cells that may succeed under circumstances in which the population vector fails. 2. Population codes are analyzed using probability theory to find the complete conditional probability density of a movement parameter given the firing pattern of a set of cells. 3. The conditional probability density when a single cell fires is proportional to the shape of the cell's tuning curve of firing rate in response to different movement parameters. 4. The conditional density when multiple cells fire is proportional to the product of their tuning curves. 5. Movement parameters can be estimated from the conditional density using statistical maximum likelihood or minimum mean-squared error methods. 6. Simulations show that density estimation correctly finds movement directions for nonuniform distributions of preferred directions and noncosine cell tuning curves, whereas the population vector method fails for these cases. 7. Probability methods thus provide a statistically based alternative to the population vector for interpreting electrophysiological recording data from multiple cells.

title: Whisker Vibration Information Carried by Rat Barrel Cortex Neurons
abstract: Rats can make extremely fine texture discriminations by “whisking” their vibrissa across the surface of an object. We have investigated one hypothesis for the neuronal basis of texture representation by measuring how clusters of neurons in the barrel cortex of anesthetized rats encode the kinetic features of sinusoidal whisker vibrations. Mutual information analyses of spike counts led to a number of findings. Information about vibration kinetics became available as early as 6 msec after stimulus onset and reached a peak at ∼20-30 msec. Vibration speed, proportional to the product of vibration amplitude (A) and frequency (f), was the kinetic property most reliably reported by cortical neurons. Indeed, by measuring information when the complete stimulus set was collapsed into feature-defined groups, we found that neurons reduced the dimensionality of the stimulus from two features (A, f) to a single feature, the product Af. Moreover, because different neurons encode stimuli in the same manner, information loss was negligible even when the activity of separate neuronal clusters was pooled. This suggests a decoding scheme whereby target neurons could capture all available information simply by summating the signals from separate barrel cortex neurons. These results indicate that neuronal population activity provides sufficient information to allow nearly perfect discrimination of two vibrations, based on their deflection speeds, within a time scale comparable with that of a single whisking motion across a surface.

title: Population coding by electrosensory neurons.
abstract: Sensory stimuli typically activate many receptors at once and therefore should lead to increases in correlated activity among central neurons. Such correlated activity could be a critical feature in the encoding and decoding of information in central circuits. Here we characterize correlated activity in response to two biologically relevant classes of sensory stimuli in the primary electrosensory nuclei, the electrosensory lateral line lobe, of the weakly electric fish Apteronotus leptorhynchus. Our results show that these neurons can display significant correlations in their baseline activities that depend on the amount of receptive field overlap. A detailed analysis of spike trains revealed that correlated activity resulted predominantly from a tendency to fire synchronous or anti-synchronous bursts of spikes. We also explored how different stimulation protocols affected correlated activity: while prey-like stimuli increased correlated activity, conspecific-like stimuli decreased correlated activity. We also computed the correlations between the variabilities of each neuron to repeated presentations of the same stimulus (noise correlations) and found lower amounts of noise correlation for communication stimuli. Therefore the decrease in correlated activity seen with communication stimuli is caused at least in part by reduced noise correlations. This differential modulation in correlated activity occurred because of changes in burst firing at the individual neuron level. Our results show that different categories of behaviorally relevant input will differentially affect correlated activity. In particular, we show that the number of correlated bursts within a given time window could be used by postsynaptic neurons to distinguish between both stimulus categories.

title: A Population Decoding Framework for Motion Aftereffects on Smooth Pursuit Eye Movements
abstract: Both perceptual and motor systems must decode visual information from the distributed activity of large populations of cortical neurons. We have sought a common framework for understanding decoding strategies for visually guided movement and perception by asking whether the strong motion aftereffects seen in the perceptual domain lead to similar expressions in motor output. We found that motion adaptation indeed has strong sequelae in the direction and speed of smooth pursuit eye movements. After adaptation with a stimulus that moves in a given direction for 7 sec, the direction of pursuit is repelled from the direction of pursuit targets that move within 90° of the adapting direction. The speed of pursuit decreases for targets that move at the direction and speed of the adapting stimulus and is repelled from the adapting speed in the sense that the decrease either becomes greater or smaller (eventually turning to an increase) when tracking targets move slower or faster than the adapting speed. The effects of adaptation are spatially specific and fixed to the retinal location of the adapting stimulus. The magnitude of adaptation of pursuit speed and direction is uncorrelated, suggesting that the two parameters are decoded independently. Computer simulation of motion adaptation in the middle temporal visual area (MT) shows that vector-averaging decoding of the population response in MT can account for the effects of adaptation on the direction of pursuit. Our results suggest a unified framework for thinking, in terms of population decoding, about motion adaptation for both perception and action.

title: Decoding the temporal evolution of a simple perceptual act.
abstract: Most perceptual tasks require sequential steps to be carried out. This must be the case, for example, when subjects discriminate the difference in frequency between two mechanical vibrations applied sequentially to their fingertips. This perceptual task can be understood as a chain of neural operations: encoding the two consecutive stimulus frequencies, maintaining the first stimulus in working memory, comparing the second stimulus to the memory trace left by the first stimulus, and communicating the result of the comparison to the motor apparatus. Where and how in the brain are these cognitive operations executed? We addressed this problem by recording single neurons from several cortical areas while trained monkeys executed the vibrotactile discrimination task. We found that primary somatosensory cortex (S1) drives higher cortical areas where past and current sensory information are combined, such that a comparison of the two evolves into a decision. Consistent with this result, direct activation of the S1 can trigger quantifiable percepts in this task. These findings provide a fairly complete panorama of the neural dynamics that underlies the transformation of sensory information into an action and emphasize the importance of studying multiple cortical areas during the same behavioural task.

title: Perceiving heading with different retinal regions and types of optic flow
abstract: We examined the ability to use optic flow to judge heading when different parts of the retina are stimulated and when the specified heading is in different directions relative to the display. To do so, we manipulated retinal eccentricity (the angle between the fovea and the center of the stimulus) and heading eccentricity (the angle between the specified heading and the center of the stimulus) independently. Observers viewed two sequences of moving dots that simulated translation through a random cloud of dots. They reported whether the direction of translation—the heading—in the second sequence was to the left or right of the direction in the first sequence. The results revealed a large and consistent effect of heading eccentricity: Judgments were much more accurate with radial flow fields (small heading eccentricities) than with lamellar fields (large heading eccentricities), regardless of the part of the retina being stimulated. The results also revealeda smaller and less consistent effect of retinal eccentricity: With radial flow (small heading eccentricities), judgments were more accurate when the stimulus was presented near the fovea. The variation of heading thresholds from radial to lamellar flow fields is predicted by a simple model of two-dimensional motion discrimination. The fact that the predictions are accurate implies that the human visual system is equally efficient at processing radial and lamellar flow fields. In addition, efficiency is reasonably constant no matter what part of the retina is being stimulated.

title: MST neurons respond to optic flow and translational movement.
abstract: We recorded the responses of 189 medial superior temporal area (MST) neurons by using optic flow, real translational movement, and combined stimuli in which matching directions of optic flow and real translational movement were presented together. One-half of the neurons (48%) showed strong responses to optic flow simulating self-movement in the horizontal plane, and 24% showed strong responses to translational movement. Combining optic flow stimuli with matching directions of translational movement caused substantial changes in both the amplitude of the best responses (44% of neurons) and the strength of direction selectivity (71% of neurons), with little effect on which stimulus direction was preferred. However, combining optic flow and translational movement such that opposite directions were presented together changed the preferred direction in 45% of the neurons with substantial changes in the strength of direction selectivity. These studies suggest that MST neurons combine visual and vestibular signals to enhance self-movement detection and disambiguate optic flow that results from either self-movement or the movement of large objects near the observer.

title: Visual and Nonvisual Contributions to Three-Dimensional Heading Selectivity in the Medial Superior Temporal Area
abstract: Robust perception of self-motion requires integration of visual motion signals with nonvisual cues. Neurons in the dorsal subdivision of the medial superior temporal area (MSTd) may be involved in this sensory integration, because they respond selectively to global patterns of optic flow, as well as translational motion in darkness. Using a virtual-reality system, we have characterized the three-dimensional (3D) tuning of MSTd neurons to heading directions defined by optic flow alone, inertial motion alone, and congruent combinations of the two cues. Among 255 MSTd neurons, 98% exhibited significant 3D heading tuning in response to optic flow, whereas 64% were selective for heading defined by inertial motion. Heading preferences for visual and inertial motion could be aligned but were just as frequently opposite. Moreover, heading selectivity in response to congruent visual/vestibular stimulation was typically weaker than that obtained using optic flow alone, and heading preferences under congruent stimulation were dominated by the visual input. Thus, MSTd neurons generally did not integrate visual and nonvisual cues to achieve better heading selectivity. A simple two-layer neural network, which received eye-centered visual inputs and head-centered vestibular inputs, reproduced the major features of the MSTd data. The network was trained to compute heading in a head-centered reference frame under all stimulus conditions, such that it performed a selective reference-frame transformation of visual, but not vestibular, signals. The similarity between network hidden units and MSTd neurons suggests that MSTd may be an early stage of sensory convergence involved in transforming optic flow information into a (head-centered) reference frame that facilitates integration with vestibular signals.

title: Heading representation in MST: sensory interactions and population encoding.
abstract: Dorsal medial superior temporal cortex (MSTd)'s population response encodes heading direction from optic flow seen during fixation or pursuit. Vestibular responses in these neurons might enhance heading representation during self-movement in light or provide an alternative basis for heading representation during self-movement in darkness. We have compared these hypotheses by recording MSTd neuronal responses to translational self-movement in light and darkness, during fixation and pursuit. Translational movement in darkness, with gaze fixed, evokes transient vestibular responses during acceleration that reverse directionality during deceleration and persist without a fixation target. Movement in light increases the amplitude and duration of these responses so they mimic responses to simulated optic flow presented without translational movement. Pursuit of a stationary landmark during translational movement combines vestibular and visual effects with pursuit responses. Vestibular, visual, and pursuit effects interact so that single neuron heading responses vary across the stimulus period and between stimulus conditions. Combining single neuron responses by population vector summation yields stronger heading estimates in light than in darkness, with gaze fixed or during landmark pursuit. Adding translational movement to robust optic flow stimuli does not augment the population response. Vestibular signals enhance single neuron responses in light and maintain population heading estimation in darkness, potentially extending MSTd's heading representation across the continuum of naturalistic self-movement conditions.

title: Multimodal Coding of Three-Dimensional Rotation and Translation in Area MSTd: Comparison of Visual and Vestibular Selectivity
abstract: Recent studies have shown that most neurons in the dorsal medial superior temporal area (MSTd) signal the direction of self-translation (i.e., heading) in response to both optic flow and inertial motion. Much less is currently known about the response properties of MSTd neurons during self-rotation. We have characterized the three-dimensional tuning of MSTd neurons while monkeys passively fixated a central, head-fixed target. Rotational stimuli were either presented using a motion platform or simulated visually using optic flow. Nearly all MSTd cells were significantly tuned for the direction of rotation in the absence of optic flow, with more neurons preferring roll than pitch or yaw rotations. The preferred rotation axis in response to optic flow was generally the opposite of that during physical rotation. This result differs sharply from our findings for translational motion, where approximately half of MSTd neurons have congruent visual and vestibular preferences. By testing a subset of neurons with combined visual and vestibular stimulation, we also show that the contributions of visual and vestibular cues to MSTd responses depend on the relative reliabilities of the two stimulus modalities. Previous studies of MSTd responses to motion in darkness have assumed a vestibular origin for the activity observed. We have directly verified this assumption by recording from MSTd neurons after bilateral labyrinthectomy. Selectivity for physical rotation and translation stimuli was eliminated after labyrinthectomy, whereas selectivity to optic flow was unaffected. Overall, the lack of MSTd neurons with congruent rotation tuning for visual and vestibular stimuli suggests that MSTd does not integrate these signals to produce a robust perception of self-rotation. Vestibular rotation signals in MSTd may instead be used to compensate for the confounding effects of rotatory head movements on optic flow.

title: Clustering of response selectivity in the medial superior temporal area of extrastriate cortex in the macaque monkey.
abstract: Ever since being described by Mountcastle (Mountcastle, 1957), columnar organization of sensory cortical areas has provided key leverage into understanding the functional organization of neocortex. Columnar or clustered organization of neurons sharing like properties is now known to be widespread, and probably universal in primary sensory areas. Visual cortex in primates consists of a primary area and a large number of secondary areas, which are organized in a manner both hierarchical and parallel (Felleman & Van Essen, 1991;  1993; Young et al., 1995). One major component in the organization of extrastriate visual cortex appears to be the division into dorsal and ventral "streams" of processing (Ungerleider & Mishkin, 1982), each of which is organized hierarchically. Within each, columnar organization exists at early stages, but becomes less clear at higher levels. Columnar organization has been described at the highest level of the ventral stream, inferotemporal cortex (IT, Saleem et al., 1993; Fujita & Fujita, 1996; Tanaka, 1996), but has not been well characterized at the higher levels of the dorsal stream. Hints of such organization are found in the literature (Saito et al., 1986; Lagae et al., 1994), but systematic measurements are needed. In this paper, I report the existence of clustered organization in the medial superior temporal area (MST) of the dorsal stream, which is arguably the highest dominantly visual area on this pathway. I have measured the selectivity of both single- and multiple-unit activity along oblique electrode penetrations through this area to three different kinds of optic flow stimuli, and find that nearby neurons are more similar in their tuning than are more distant ones. This observation documents the existence of some form of clustered organization and supports the importance of this area in the processing of optic flow information.

title: Area MST and heading perception in macaque monkeys.
abstract: The macaque medial superior temporal area (MST) is proposed to be specialized for analyzing complex 'optic flow' information. Such space-varying motion patterns provide a rich source of information about self motion, scene structure and object shape. We report the performance of rhesus macaques on a two-alternative 'heading' task, in which they reported whether horizontally varying, simulated trajectories were to left or right of center. Monkeys were sensitive to small heading angles; thresholds averaged 1.5-3 degrees. Heading estimates were stable in the face of changing stimulus location and smooth pursuit eye movements. In addition, we tested the role of area MST in heading judgements by electrically activating columns of neurons in this area while the monkeys performed the heading task. Activation of MST frequently affected performance, usually causing choice biases. These induced biases were often large and usually concordant with the preference of the neurons being activated. In addition, the induced biases were often larger in the presence of smooth pursuit eye movements. These results favor the hypothesis that MST is involved in recovering self-motion direction from optic flow cues and in the process by which heading perception is compensated for ongoing eye movements.

title: A functional link between area MSTd and heading perception based on vestibular signals
abstract: Recent findings of vestibular responses in part of the visual cortex—the dorsal medial superior temporal area (MSTd)—indicate that vestibular signals might contribute to cortical processes that mediate the perception of self-motion. We tested this hypothesis in monkeys trained to perform a fine heading discrimination task solely on the basis of inertial motion cues. The sensitivity of the neuronal responses was typically lower than that of psychophysical performance, and only the most sensitive neurons rivaled behavioral performance. Responses recorded in MSTd were significantly correlated with perceptual decisions, and the correlations were strongest for the most sensitive neurons. These results support a functional link between MSTd and heading perception based on inertial motion cues. These cues seem mainly to be of vestibular origin, as labyrinthectomy produced a marked elevation of psychophysical thresholds and abolished MSTd responses. This study provides evidence that links single-unit activity to spatial perception mediated by vestibular signals, and supports the idea that the role of MSTd in self-motion perception extends beyond optic flow processing.

title: Neural correlates of multisensory cue integration in macaque MSTd
abstract: Human observers combine multiple sensory cues synergistically to achieve greater perceptual sensitivity, but little is known about the underlying neuronal mechanisms. We recorded the activity of neurons in the dorsal medial superior temporal (MSTd) area during a task in which trained monkeys combined visual and vestibular cues near-optimally to discriminate heading. During bimodal stimulation, MSTd neurons combined visual and vestibular inputs linearly with subadditive weights. Neurons with congruent heading preferences for visual and vestibular stimuli showed improvements in sensitivity that parallel behavioral effects. In contrast, neurons with opposite preferences showed diminished sensitivity under cue combination. Responses of congruent cells were more strongly correlated with monkeys' perceptual decisions than were responses of opposite cells, suggesting that the monkey monitored the activity of congruent cells to a greater extent during cue integration. These findings show that perceptual cue integration occurs in nonhuman primates and identify a population of neurons that may form its neural basis.

title: Response of monkey MST neurons to optic flow stimuli with shifted centers of motion
abstract: Neurons in the dorsal region of the medial superior temporal area (MSTd) have previously been shown to respond to the expanding radial motion that occurs as an observer moves through the environment. In previous experiments, MSTd neurons were tested with radial and circular motion centered in the visual field. However, different directions of observer motion, relative to the direction of gaze, are accompanied by visual motion centered at different locations in the visual field. The present experiments investigated whether neurons that respond to radial and circular motion might respond differently when the center of motion was shifted to different regions of the visual field. About 90% of the 245 neurons studied responded differently when the center of motion was shifted away from the center of the field. The centers of motion preferred by each neuron were limited to one area of the visual field. All parts of the visual field were represented in the sample, with greater numbers of neurons preferring centers of motion closer to the center of the field. We hypothesize that each of the MSTd neurons has a center of motion field with a gradient of preferred centers of motion, and that there is an orderly arrangement of MSTd neurons with each region of the visual field being represented by a set of neurons. This arrangement creates the potential for graded responses from individual neurons for different directions of heading as an observer moves through the environment.

title: Optimal representation of sensory information by neural populations
abstract: Sensory information is encoded by populations of neurons. The responses of individual neurons are inherently noisy, so the brain must interpret this information as reliably as possible. In most situations, the optimal strategy for decoding the population signal is to compute the likelihoods of the stimuli that are consistent with an observed neural response. But it has not been clear how the brain can directly compute likelihoods. Here we present a simple and biologically plausible model that can realize the likelihood function by computing a weighted sum of sensory neuron responses. The model provides the basis for an optimal decoding of sensory information. It explains a variety of psychophysical observations on detection, discrimination and identification, and it also directly predicts the relative contributions that different sensory neurons make to perceptual judgments.

title: Neural population code for fine perceptual decisions in area MT
abstract: In the middle temporal (MT) area of primates, many motion-sensitive neurons with a wide range of preferred directions respond to a stimulus moving in a single direction. These neurons are involved in direction perception, but it is not clear how perceptual decisions are related to the population response. We recorded the activities of MT neurons in rhesus monkeys while they discriminated closely related directions, and examined the relationship between the activities of neurons tuned to different directions and the monkeys' choices. Perceptual decisions were significantly correlated with the activities of the highest-precision neurons but not with those of the lowest-precision neurons. The combined performance of the high-precision neurons matched the monkeys' behavior, whereas the ability to predict behavior based on the entire active population was poor. These results suggest that fine discrimination decisions are crucially dependent on the activities of the most informative neurons.

title: A Logarithmic, Scale-Invariant Representation of Speed in Macaque Middle Temporal Area Accounts for Speed Discrimination Performance
abstract: Human speed discrimination thresholds follow Weber's law over a large range of reference (i.e., pedestal) speeds, that is, the just-noticeable-difference in speed scales in proportion to the reference speed. We analyzed the neural representation of speed information in macaque middle temporal visual area (MT) to determine whether this representation can account for the basic form of psychophysical data. Based on theoretical considerations, we hypothesized: (1) that the speed tuning curves of MT neurons should be bell-shaped (Gaussian) as a function of the logarithm of speed, (2) that the set of speed-tuning curves should be approximately scale-invariant, (3) that the distribution of speed preferences should be approximately uniform in log speed, and (4) that response variability should be independent of speed preference. Our quantitative analysis of data from 501 MT neurons shows that the neural representation of speed approximately obeys these constraints, with modest deviations particularly at slow speeds. We then used the MT data to predict how speed discrimination thresholds should depend on pedestal speed. The shape of this prediction matches very closely to that of human psychophysical data, accounting for constant Weber fractions over a large range of intermediate speeds as well as a marked departure from Weber's law at slow speeds. Moreover, we show that deviations of the MT representation from the above constraints are important for predicting how psychophysical thresholds depart from Weber's law at slow speeds. These findings support the notion that a logarithmic, approximately scale-invariant representation of speed in area MT limits perceptual speed discrimination.

title: Neuronal population coding of movement direction.
abstract: Although individual neurons in the arm area of the primate motor cortex are only broadly tuned to a particular direction in three-dimensional space, the animal can very precisely control the movement of its arm. The direction of movement was found to be uniquely predicted by the action of a population of motor cortical neurons. When individual cells were represented as vectors that make weighted contributions along the axis of their preferred direction (according to changes in their activity during the movement under consideration) the resulting vector sum of all cell vectors (population vector) was in a direction congruent with the direction of movement. This population vector can be monitored during various tasks, and similar measures in other neuronal populations could be of heuristic value where there is a neural representation of variables with vectorial attributes.

title: Perception of translational heading from optical flow.
abstract: Radial patterns of optical flow produced by observer translation could be used to perceive the direction of self-movement during locomotion, and a number of formal analyses of such patterns have recently appeared. However, there is comparatively little empirical research on the perception of heading from optical flow, and what data there are indicate surprisingly poor performance, with heading errors on the order of 5 degrees-10 degrees. We examined heading judgments during translation parallel, perpendicular, and at oblique angles to a random-dot plane, varying observer speed and dot density. Using a discrimination task, we found that heading accuracy improved by an order of magnitude, with 75%-correct thresholds of 0.66 degrees in the highest speed and density condition and 1.2 degrees generally. Performance remained high with displays of 63-10 dots, but it dropped significantly with only 2 dots; there was no consistent speed effect and no effect of angle of approach to the surface. The results are inconsistent with theories based on the local focus of outflow, local motion parallax, multiple fixations, differential motion parallax, and the local maximum of divergence. But they are consistent with Gibson's (1950) original global radial outflow hypothesis for perception of heading during translation.

title: Visual self-motion perception during head turns
abstract: Extra-retinal information is critical in the interpretation of visual input during self-motion. Turning our eyes and head to track objects displaces the retinal image but does not affect our ability to navigate because we use extra-retinal information to compensate for these displacements. We showed observers animated displays depicting their forward motion through a scene. They perceived the simulated self-motion accurately while smoothly shifting the gaze by turning the head, but not when the same gaze shift was simulated in the display; this indicates that the visual system also uses extra-retinal information during head turns. Additional experiments compared self-motion judgments during active and passive head turns, passive rotations of the body and rotations of the body with head fixed in space. We found that accurate perception during active head turns is mediated by contributions from three extra-retinal cues: vestibular canal stimulation, neck proprioception and an efference copy of the motor command to turn the head.

title: Eye movements and optical flow.
abstract: Translation of an observer through a static environment generates a pattern of optical flow that specifies the direction of self-motion, but the retinal flow pattern is confounded by pursuit eye movements. How does the visual system decompose the translational and rotational components of flow to determine heading? It is shown that observers can perceive their direction of self-motion during stationary fixations and pursuit eye movements and with displays that simulate the optical effects of eye movements. Results indicate that the visual system can perform the decomposition with both continuous and discontinuous fields on the basis of flow-field information alone but requires a three-dimensional environmental structure to do so. The findings are inconsistent with general computational models and theories based on the maximum of divergence, oculomotor signals, or multiple fixations but are consistent with the theory of reliance on differential motion produced by environmental variation in depth.

title: Population coding of saccadic eye movements by neurons in the superior colliculus
abstract: The deeper layers of the superior colliculus are involved in the initiation and execution of saccadic (high velocity) eye movements1. A large population of coarsely tuned collicular neurons is active before each saccade. The mechanisms by which the signals that precisely control the direction and amplitude of a saccade are extracted from the activity of the population are unknown. It has been assumed26 that the exact trajectory of a saccade is determined by the activity of the entire population and that information is not extracted from only the most active cells in the population at a subsequent stage of neural processing. The trajectory of a saccade could be based on vector summation of the movement tendencies provided by each member of the population of active neurons4 or be determined by a weighted average of the vector contributions of each neuron in the active population2. Here we present the results of experiments in which a small subset of the active population was reversibly deactivated with lidocaine. These results are consistent with the predictions of the latter population-averaging hypothesis and support the general idea that the direction, amplitude and velocity of saccadic eye movements are based on the responses of the entire population of cells active before a saccadic eye movement.

title: Correlated Firing in Macaque Visual Area MT: Time Scales and Relationship to Behavior
abstract: We studied the simultaneous activity of pairs of neurons recorded with a single electrode in visual cortical area MT while monkeys performed a direction discrimination task. Previously, we reported the strength of interneuronal correlation of spike count on the time scale of the behavioral epoch (2 sec) and noted its potential impact on signal pooling (Zohary et al., 1994). We have now examined correlation at longer and shorter time scales and found that pair-wise cross-correlation was predominantly short term (10–100 msec). Narrow, central peaks in the spike train cross-correlograms were largely responsible for correlated spike counts on the time scale of the behavioral epoch. Longer-term (many seconds to minutes) changes in the responsiveness of single neurons were observed in auto-correlations; however, these slow changes in time were on average uncorrelated between neurons. Knowledge of the limited time scale of correlation allowed the derivation of a more efficient metric for spike count correlation based on spike timing information, and it also revealed a potential relative advantage of larger neuronal pools for shorter integration times. Finally, correlation did not depend on the presence of the visual stimulus or the behavioral choice of the animal. It varied little with stimulus condition but was stronger between neurons with similar direction tuning curves. Taken together, our results strengthen the view that common input, common stimulus selectivity, and common noise are tightly linked in functioning cortical circuits.

title: Noise correlations in cortical area MT and their potential impact on trial-by-trial variation in the direction and speed of smooth-pursuit eye movements.
abstract: Smooth-pursuit eye movements are variable, even when the same tracking target motion is repeated many times. We asked whether variation in pursuit could arise from noise in the response of visual motion neurons in the middle temporal visual area (MT). In physiological experiments, we evaluated the mean, variance, and trial-by-trial correlation in the spike counts of pairs of simultaneously recorded MT neurons. The correlations between responses of pairs of MT neurons are highly significant and are stronger when the two neurons in a pair have similar preferred speeds, directions, or receptive field locations. Spike count correlation persists when the same exact stimulus form is repeatedly presented. Spike count correlations increase as the analysis window increases because of correlations in the responses of individual neurons across time. Spike count correlations are highest at speeds below the preferred speeds of the neuron pair and increase as the contrast of a square-wave grating is decreased. In computational analyses, we evaluated whether the correlations and variation across the population response in MT could drive the observed behavioral variation in pursuit direction and speed. We created model population responses that mimicked the mean and variance of MT neural responses as well as the observed structure and amplitude of noise correlations between pairs of neurons. A vector-averaging decoding computation revealed that the observed variation in pursuit could arise from the MT population response, without postulating other sources of motor variation.

title: Correlated neuronal discharge rate and its implications for psychophysical performance
abstract: SINGLE neurons can signal subtle changes in the sensory environment with surprising fidelity, often matching the perceptual sensitivity of trained psychophysical observers1–10. This similarity poses an intriguing puzzle: why is psychophysical sensitivity not greater than that of single neurons? Pooling responses across neurons should average out noise in the activity of single cells, leading to substantially improved psychophysical performance. If, however, noise is correlated among these neurons, the beneficial effects of pooling would be diminished10–12. To assess correlation within a pool, the responses of pairs of neurons were recorded simultaneously during repeated stimulus presentations. We report here that the observed covariation in spike count was relatively weak, the correlation coefficient averaging 0.12. A theoretical analysis revealed, however, that weak correlation can limit substantially the signalling capacity of the pool. In addition, theory suggests a relationship between neuronal responses and psychophysical decisions which may prove useful for identifying cell populations underlying specific perceptual capacities.

title: Interaction of linear vestibular and visual stimulation in the macaque ventral intraparietal area (VIP)
abstract: Navigation in space requires the brain to combine information arising from different sensory modalities with the appropriate motor commands. Sensory information about self‐motion in particular is provided by the visual and the vestibular system. The macaque ventral intraparietal area (VIP) has recently been shown to be involved in the processing of self‐motion information provided by optical flow, to contain multimodal neurons and to receive input from areas involved in the analysis of vestibular information. By studying responses to linear vestibular, visual and bimodal stimulation we aimed at gaining more insight into the mechanisms involved in multimodal integration and self‐motion processing. A large proportion of cells (77%) revealed a significant response to passive linear translation of the monkey. Of these cells, 59% encoded information about the direction of self‐motion. The phase relationship between vestibular stimulation and neuronal responses covered a broad spectrum, demonstrating the complexity of the spatio‐temporal pattern of vestibular information encoded by neurons in area VIP. For 53% of the direction‐selective neurons the preferred directions for stimuli of both modalities were the same; they were opposite for the remaining 47% of the neurons. During bimodal stimulation the responses of neurons with opposite direction selectivity in the two modalities were determined either by the visual (53%) or the vestibular (47%) modality. These heterogeneous responses to unimodal and bimodal stimulation might be used to prevent misjudgements about self‐ and/or object‐motion, which could be caused by relying on information of one sensory modality alone.

title: Physiology of peripheral neurons innervating otolith organs of the squirrel monkey. II. Directional selectivity and force-response relations.
abstract: 1. The directional selectivity of peripheral otolith neurons was studied in the barbiturate-anesthetized squirrel monkey (Saimiri sciureus). Each unit's most sensitive axis was characterized by a functional polarization vector. The direction of a centrifugal force was varied with respect to the vector and to the macular plane. The neurons respond in an excitatory manner to shearing forces orthogonally disposed to the vector. The sensitivity to orthogonal shears was usually 10-15% of the sensitivity to parallel shearing forces. There was no significant response to orthogonal compressions, nor did compressions modify the response to shearing forces. 2. Force-response functions were obtained in the stimulus range of +/- 4.92 g. Forces were directed parallel to each unit's polarization vector. The functions are sigmoid shaped and possess both inhibitory and excitatory plateaus. The presumed physiological range of +/-1 g is represented in the lower (concave upward) portion of the function and has a dynamic range, expressed in terms of response magnitude, amounting to 20-40% of the potential dynamic range of the neuron. 3. There was considerable variation among units in their +/- 4.92 g force-response curves. The salient features of the functions are described by three factors, tentatively identified as a transduction gain, a receptor bias, and a mechanical gain. Both the resting discharge (do) and the +/-1 g sensitivity (so) vary in the same direction with changes in the factors related to transduction gain and receptor bias. It is shown that this covariation provides a quantitatively precise explanation for the positive relation between do and so.

title: Encoding of head acceleration in vestibular neurons. I. Spatiotemporal response properties to linear acceleration.
abstract: 1. Extracellular recordings were made in and around the medial vestibular nuclei in decerebrated rats. Neurons were functionally identified according to their semicircular canal input on the basis of their responses to angular head rotations around the yaw, pitch, and roll head axes. Those cells responding to angular acceleration were classified as either horizontal semicircular canal-related (HC) or vertical semicircular canal-related (VC) neurons. The HC neurons were further characterized as either type I or type II, depending on the direction of rotation producing excitation. Cells that lacked a response to angular head acceleration, but exhibited sensitivity to a change in head position, were classified as purely otolith organ-related (OTO) neurons. All vestibular neurons were then tested for their response to sinusoidal linear translation in the horizontal head plane. 2. Convergence of macular and canal inputs onto central vestibular nuclei neurons occurred in 73% of the type I HC, 79% of the type II HC, and 86% of the VC neurons. Out of the 223 neurons identified as receiving macular input, 94 neurons were further studied, and their spatiotemporal response properties to sinusoidal stimulation with pure linear acceleration were quantified. Data were obtained from 33 type I HC, 22 type II HC, 22 VC, and 17 OTO neurons. 3. For each neuron the angle of the translational stimulus vector was varied by 15, 30, or 45 degrees increments in the horizontal head plane. In all tested neurons, a direction of maximum sensitivity was identified. An interesting difference among neurons was their response to translation along the direction perpendicular to that that produced the maximum response ("null" direction). For the majority of neurons tested, it was possible to evoke a nonzero response during stimulation along the null direction always had response phases that varied as a function of stimulus direction. 4. These spatiotemporal response properties were quantified in two independent ways. First, the data were evaluated on the basis of the traditional one-dimensional principle governed by the "cosine gain rule" and constant response phase at different stimulus orientations. Second, the response gain and phase values that were empirically determined for each orientation of the applied linear stimulus vector were fitted on the basis of a newly developed formalism that treats neuronal responses as exhibiting two-dimensional spatial sensitivity. Thus two response vectors were determined for each neuron on the basis of its response gain and phase at different stimulus directions in the horizontal head plane.(ABSTRACT TRUNCATED AT 400

title: Vestibular convergence patterns in vestibular nuclei neurons of alert primates.
abstract: Sensory signal convergence is a fundamental and important aspect of brain function. Such convergence may often involve complex multidimensional interactions as those proposed for the processing of otolith and semicircular canal (SCC) information for the detection of translational head movements and the effective discrimination from physically congruent gravity signals. In the present study, we have examined the responses of primate rostral vestibular nuclei (VN) neurons that do not exhibit any eye movement-related activity using 0.5-Hz translational and three-dimensional (3D) rotational motion. Three distinct neural populations were identified. Approximately one-fourth of the cells exclusively encoded rotational movements (canal-only neurons) and were unresponsive to translation. The canal-only central neurons encoded head rotation in SCC coordinates, exhibited little orthogonal canal convergence, and were characterized with significantly higher sensitivities to rotation as compared to primary SCC afferents. Another fourth of the neurons modulated their firing rates during translation (otolith-only cells). During rotations, these neurons only responded when the axis of rotation was earth-horizontal and the head was changing orientation relative to gravity. The remaining one-half of VN neurons were sensitive to both rotations and translations (otolith + canal neurons). Unlike primary otolith afferents, however, central neurons often exhibited significant spatiotemporal (noncosine) tuning properties and a wide variety of response dynamics to translation. To characterize the pattern of SCC inputs to otolith + canal neurons, their rotational maximum sensitivity vectors were computed using exclusively responses during earth-vertical axis rotations (EVA). Maximum sensitivity vectors were distributed throughout the 3D space, suggesting strong convergence from multiple SCCs. These neurons were also tested with earth-horizontal axis rotations (EHA), which would activate both vertical canals and otolith organs. However, the recorded responses could not be predicted from a linear combination of EVA rotational and translational responses. In contrast, one-third of the neurons responded similarly during EVA and EHA rotations, although a significant response modulation was present during translation. Thus this subpopulation of otolith + canal cells, which included neurons with either high- or low-pass dynamics to translation, appear to selectively ignore the component of otolith-selective activation that is due to changes in the orientation of the head relative to gravity. Thus contrary to primary otolith afferents and otolith-only central neurons that respond equivalently to tilts relative to gravity and translational movements, approximately one-third of the otolith + canal cells seem to encode a true estimate of the translational component of the imposed passive head and body movement.

title: Responses to head tilt in cat central vestibular neurons. I. Direction of maximum sensitivity.
abstract: Responses to head tilt were recorded from vestibular neurons in and around the lateral vestibular nucleus (LVN) of the decerebrate cat. Each animal had all six semicircular canals rendered nonfunctional by a plugging procedure. Each cell was studied by slowly tilting the cat, using one or both of two paradigms. In the first method, sinusoidal tilts (0.05 or 0.1 Hz) were used to produce bidirectional stimuli in up to 12 pairs of directions, including left/right (roll tilt) and fore/aft (pitch). The second method imposed a constant 10 degree tilt; the direction of the tilt was rotated around the animal by an appropriate combination of roll and pitch motions. Neurons responded by maximally increasing their discharge frequency in a particular direction of head tilt from the horizontal. Each cell's response could be described by a vector in the animal's horizontal plane whose orientation is given by the direction of the most effective stimulus and whose length represents the neuron's maximal sensitivity to tilt. The two methods of stimulation yielded equivalent response vectors. Response vectors were obtained for 100 neurons. The distribution of vector directions for these vestibular neurons was not uniform; there was a conspicuous absence of neurons with fore/aft-directed vectors. The sensitivity of these cells (length of the response vector) ranged from 10 to 230 impulses X s-1 X g-1 (median 50). Neurons whose vectors lay in the ipsilateral half-plane (which would be excited by ear-down tilt) tended to be less sensitive than those with contralateral vectors. Neurons excited by ear-up tilt tended to be located ventrally in the LVN, while those excited by ear-down tilt were more evenly distributed. There was no other obvious correlation of vector orientation with the anatomical locus of the cell in the LVN. The directional selectivity of the responses of these neurons to head tilts are similar to those previously reported tin utricular afferents. The broad distribution of response vector orientations provides an appropriate substrate for vestibulospinal reflexes to a wide variety of head tilts.

title: Properties of cerebellar fastigial neurons during translation, rotation, and eye movements.
abstract: The most medial of the deep cerebellar nuclei, the fastigial nucleus (FN), receives sensory vestibular information and direct inhibition from the cerebellar vermis. We investigated the signal processing in the primate FN by recording single-unit activities during translational motion, rotational motion, and eye movements. Firing rate modulation during horizontal plane translation in the absence of eye movements was observed in all non-eye-movement-sensitive cells and 26% of the pursuit eye-movement-sensitive neurons in the caudal FN. Many non-eye-movement-sensitive cells recorded in the rostral FN of three fascicularis monkeys exhibited convergence of signals from both the otolith organs and the semicircular canals. At low frequencies of translation, the majority of these rostral FN cells changed their firing rates in phase with head velocity rather than linear acceleration. As frequency increased, FN vestibular neurons exhibited a wide range of response dynamics with most cells being characterized by increasing phase leads as a function of frequency. Unlike cells in the vestibular nuclei, none of the rostral FN cells responded to rotational motion alone, without simultaneously exhibiting sensitivity to translational motion. Modulation during earth-horizontal axis rotation was observed in more than half (77%) of the neurons, although with smaller gains than during translation. In contrast, only 47% of the cells changed their firing rates during earth-vertical axis rotations in the absence of a dynamic linear acceleration stimulus. These response properties suggest that the rostral FN represents a main processing center of otolith-driven information for inertial motion detection and spatial orientation.

title: The psychometric function: I. Fitting, sampling, and goodness of fit
abstract: The psychometric function relates an observer’s performance to an independent variable, usually some physical quantity of a stimulus in a psychophysical task. This paper, together with its companion paper (Wichmann & Hill, 2001), describes an integrated approach to (1) fitting psychometric functions, (2) assessing the goodness of fit, and (3) providing confidence intervals for the function’s parameters and other estimates derived from them, for the purposes of hypothesis testing. The present paper deals with the first two topics, describing a constrained maximum-likelihood method of parameter estimation and developing several goodness-of-fit tests. Using Monte Carlo simulations, we deal with two specific difficulties that arise when fitting functions to psychophysical data. First, we note that human observers are prone to stimulus-independent errors (orlapses). We show that failure to account for this can lead to serious biases in estimates of the psychometric function’s parameters and illustrate how the problem may be overcome. Second, we note that psychophysical data sets are usually rather small by the standards required by most of the commonly applied statistical tests. We demonstrate the potential errors of applying traditionalX2 methods to psychophysical data and advocate use of Monte Carlo resampling techniques that do not rely on asymptotic theory. We have made available the software to implement our methods.

title: Thresholds for the detection of the direction of whole-body, linear movement in the horizontal plane.
abstract: Thresholds for the detection (at p = 0.67 correct) of the direction of discrete linear movements in the horizontal plane, having a cosine bell velocity trajectory and duration of 3 s, were determined in 24 subjects. Thresholds in the Z body axis (mean 0.154 m X s-2) were significantly higher than thresholds for movement in the X (mean 0.063 m X s-2) and Y (mean 0.057 m X s-2) body axes. In 8 subjects, X axis acceleration threshold was found to increase as a monotonic function of stimulus duration over the range 0.98 to 6.96 s and exhibited similar frequency-dependent characteristics to thresholds for the detection of continuous oscillatory stimuli. This finding implies that the sensory system mediating the transduction and perception of liminal, whole-body linear movement is sensitive to a combination of the acceleration and rate change of acceleration (jerk) of the motion stimulus, and has similar dynamics to the "irregular" sensory receptors of the otolith organs.

title: Thresholds for perception of direction of linear acceleration as a possible evaluation of the otolith function
abstract: BackgroundPrevious attempts to measure otolith function using ocular counter-rolling have shown poor sensitivity and specificity, thereby hindering a useful clinical application. We have conducted a study to investigate whether thresholds for the perception of the direction of linear acceleration might be an alternative for the clinical evaluation of otolith or statolith function.MethodsPerception of the direction of motion was evaluated in 28 healthy subjects while all external auditory and visual cues were eliminated. Whole body motion stimulus was generated by a motor driven linear sled at a stimulus frequency of 1 Hz at a linear acceleration ranging from 0 to maximum 40 cm/ s2. Subjects were required to correctly indicate the direction of motion (anterior-posterior or lateral) or whether they were stationary. Both velocity and acceleration thresholds were measured.ResultsThe median acceleration thresholds for the perception of direction of linear movement for anterior-posterior movement was 8.5 cm/s2 and for lateral movement 6.5 cm/s2. According to the literature, acceleration thresholds depend on the stimulus profile whereas velocity thresholds do not. The median velocity thresholds for the perception of direction of linear movement for anterior-posterior movement was 13.5 cm/s and for lateral movement was 10.4 cm/s. The median velocity thresholds for the perception of direction of linear movement for anterior-posterior movement increased linearly with age, whereas the median velocity threshold for lateral movement was not correlated with age.ConclusionThe thresholds found in this study are lower than reported in the literature before which may be due to the repetative predictive sinusoidal stimulus which makes it relatively easy to lower the threshold by learning already within one test prophile.The variablity is large in line with the previous literature, but our experiments indicate that variability decreases after a training session. We interprete the literature and our current results that linear velocity thresholds after some training might reflect the sensitivity of the otolith system per se.