322

J. Opt. Soc. Am. A/Vol. 2, No. 2/February 1985

A. B. Watson and A. J. Ahumada, Jr.

Model of human visual-motion sensing
Andrew B. Watson and Albert J.Ahumada, Jr.
Perception and Cognition Group, NASA Ames Research Center, Moffett Field, California 94035
Received August 15, 1984; Accepted October 22, 1984
We propose a model of how humans sense the velocity of moving images. The model exploits constraints provided
by human psychophysics, notably that motion-sensing elements appear tuned for two-dimensional spatial frequency, and by the frequency spectrum of a moving image, namely, that its support lies in the plane in which the temporal frequency equals the dot product of the spatial frequency and the image velocity. The first stage of the model
is a set of spatial-frequency-tuned, direction-selective linear sensors. The temporal frequency of the response ofeach sensor is shown to encode the component of the image velocity in the sensor direction. At the second stage,these components are resolved in order to measure the velocity of image motion at each of a number of spatial locations and spatial frequencies. The model has been applied to several illustrative examples, including apparent motion, coherent gratings, and natural image sequences. The model agrees qualitatively with human perception.

INTRODUCTION
Visual motion perception serves many roles, among them theperception of depth, the segregation of objects, the control ofsome eye movements, and the estimation of the motion of
objects in the world. Although it is clearly important, thevisual-motion sense does not have a widely agreed on principleof operation. One view is that the brain detects a physicalproperty of the stimulus, which we may call movement,
sometimes also called optical flow.' When this property isdetected, the observer has a sensation of visual motion, which
may be attached to some visual object. Occasionally, motion
is sensed in stimuli that do not contain motion, in which casewe speak of apparent motion. An alternative and we believecorrect view is that movement is not a physical property of thestimulus. Like beauty and color, motion is in the eye of thebeholder. Images do not move but rather change in many andvarious ways. Some of these changes give rise to an impression of movement. In short, all movement is apparent.
Both of the preceding points of view agree that motion
perception is a process that examines the physical stimulus
and controls perceptions, judgments, and actions related tothe motion of objects or their projections. We adopt the term'
motion sensor for a mechanism at the front end of this process.
It begins with the physical stimulus and ends with some explicit representation of quantities related to motion of either
the image or the world.
From the optical-flow point of view, the job of the motion
sensor is to measure the motion in the stimulus. A model ofthe sensor is easily constructed, since it is given directly by thephysical definition of the quantity to be detected. But, if
motion is not a physical property of the stimulus, what is thefunction of the sensor? It is to sense physical features of thestimulus that are likely (but not certain) to be informative
about the motion of objects in the world. The latter case ismuch more difficult for vision theory. There are many candidate features, and, to model the sensor, we must select aparticular set. There are two ways of making this selection.
The first is to appeal to invariants in the imagery projectedby objects in motion. 2 -4 The second approach is to identify
a set of features that make sense of the known properties of
0740-3232/85/020322-21$02.00

human vision.5 In this paper we make use of both approaches.
The plan of this paper is as follows: Section 1 summarizes
some properties of human motion perception. In Section 2,
we examine the frequency representation of a moving image
and propose a set of features to be used by the motion sensor.
In Sections 3 and 4, we develop our model of the human motion sensor. An implementation of the model is described in
Section 5, and in Section 6 we show some preliminary simulations. In Section 7, we make some concluding comments
and suggest how the model may be used to analyze and direct
psychophysical, physiological, and computational research.
1. PROPERTIES OF HUMAN MOTIONPERCEPTIONIn this section, we note some of the important properties ofhuman motion perception. These will guide and motivate theconstruction of our motion sensor.
A. Stimulus for Motion
The stimulus for visual motion is primarily the distributionof light over time and over two spatial dimensions in a plane
before the eyes. Beyond this point, what happens to thesignal is sufficiently obscure that a model must be explicit. It
is unreasonable for a model to begin with quantities such asedges or optical flow without being explicit about how thesequantities are to be computed, or demonstrating that they
could be computed, or at least acknowledging that this partof the problem is being sidestepped. In the model developed
here, we begin with a distribution of contrast over dimensionsof x, y, and t. A digital version of this distribution can beobtained by sampling the output of a video camera.
B. Things Appear to Move with Direction and Speed
Everyday visual experience shows that humans see things
move and can with some accuracy judge how fast and in what
direction they move. Thus the representation of the image
produced by the model should include an explicit assignmentof velocity. Whatever other details of the visual process we© 1985 Optical Society of America

A. B. Watson and A. J. Ahumada, Jr.

may try to simulate, our model will have little value if it fails
to meet this first test.
C. Perceived Motion Is Local
We are capable of seeing different motions in different partsof an image, such as when two objects move relative to eachother. The motion sensor must therefore assign different
velocities to different local regions of the visual field. The
local assignment of velocity implies that each sensor must
respond selectively to a local region of the visual field. There
are few data about how local these operations must be, andthis is a promising area for further research.
D. Perceived Motion Is Specific to Spatial Frequency
There are several pieces of evidence that show that individual
motion sensors respond selectively to different bands of spatial frequency. Perhaps the most compelling is an experiment
by Adelson and Movshon 6 in which they superimposed moving sinusoidal gratings of different orientations. When thetwo gratings were similar in spatial frequency, the percept wasof a single coherent plaid pattern with an apparent velocity
equal to that of the nodes of the pattern. When the spatial
frequencies differed by an octave or so, the two gratings appeared to move independently past each other in their own
separate directions.
Another result in this vein is the observation that apparent
motion is spatial-frequency specific. 7 Two small patches ofspatial grating were presented briefly at nearby positions andagain briefly with their positions exchanged. This crossed
phi sets up a competition between the perception of patches
that remain in static positions and the perception of patches
that move and exchange positions. When the spatial
frequencies of the two patches are the same, there is no apparent movement. This is not surprising, since nothing haschanged from the first presentation to the second. However,when the two spatial frequencies differ by an octave or so, thepatches appear to move across each other and exchange positions. If apparent motion were not spatial-frequency specific, there would be no reason for the patches to move. It is
not the case that any difference between the two patchessuffices to induce apparent motion. For example, if thepatches are of the same spatial frequency but different orientation, there is no apparent motion between the patches.
Numerous experiments have shown that the mechanismsthat detect stationary contrast are selective for spatial frequency.81 0 This also appears to be so for moving-contrast
patterns." It has also been shown that the mechanisms that
detect moving contrast are direction selective (see Subsection
1.G) and signal the direction of motion (at least to within 90deg) (see Subsection 1.H), properties that are symptomaticof a motion sensor. Taken together, these results suggest thatthe detectors of moving contrast are spatial-frequency specific.
To summarize, the motion sensor is selective for spatial
frequency and may assign different velocities to different
spatial-frequency bands. The computation of velocity appears to be carried out independently at a number of different
spatial scales.
E. Brief Exposures
Brief exposures to moving stimuli are sufficient to produce
vivid motion sensations and accurate motion judgments. For

Vol. 2, No. 2/February 1985/J. Opt. Soc. Am. A

323

example, at threshold contrast, observers correctly judge thedirection (left or right) of a moving grating when the exposure
duration is only 410 msec.' 2 Increasing the duration by afactor of 5 has little effect.' 3 Above threshold, a 5%differencein the speed of a moving line can be discriminated for exposures as short as 200 msec.14 Evidently, the spatiotemporalfeature extracted by the human motion sensor must be brief.
Put another way, the sensor must integrate over only a brief
interval of time.
F. Adaptation to Motion
Prolonged viewing of a pattern that moves in one direction can
produce large alterations in the visibility of subsequently
viewed stimuli. These effects are of three general kinds, depending on the type of test stimulus viewed after adaptation.
If the test stimulus is in fact stationary, it may appear to movein a direction opposite to the adapting pattern.15 This is thewell-known motion aftereffect. If the test stimulus moves ina direction similar to that of the adapting pattern, its apparent
direction may be repulsed away from the adapting direction
by as much as 10 deg.16 And, finally, contrast thresholds areraised much more for test stimuli that move in the adapting
direction than for those that move in the opposite direction.' 7
All these effects suggest the existence of separate, adaptable, direction-selective sensors. They further suggest thatthe final estimate of velocity is the result of computations
across a population of sensors selective for different directions.
G. Subthreshold Summation
The threshold contrast for a sinusoidal grating moving to theright is almost unaffected by a grating of equal contrast
moving to the left.12 18 19 This is so in spite of the fact thatthe sum of two gratings that move in opposite directions has
twice the peak contrast of either grating alone. The result
indicates that the mechanisms that detect moving patterns
are direction selective, in the sense that they are sensitive toless than the full 360-deg range of movement directions. It
also indicates that, at least near threshold, the detectors of
opposite directions are independent of one another.
This result is observed only when the image velocity isgreater than about 1 deg/sec.12"19 Below this point, considerable summation is observed between gratings that move in
opposite directions, consistent with detection by nondirection-selective mechanisms. This agrees with other evidence
that below this point a separate, nondirection-selective, static
system is more sensitive (see Subsection 1.K).
When two gratings of the same spatial frequency u moving
in opposite directions at speed r are added together, the result
is a stationary grating whose contrast varies sinusoidally intime (flickers) with a temporal frequency of w = ru and whose
contrast is twice that of either moving component. The data
cited indicated that at low velocities the thresholds for moving
and flickering gratings of the same spatial and temporal frequency are about equal, while at higher velocities sensitivity
is almost twice as great for the moving grating. Over the fullrange of velocities, therefore, sensitivities to these two patterns
are equal to within a factor of 2.
H. Direction Discrimination at Threshold
Observers are able to tell which way things are going at thethreshold of vision. More precisely, the contrast threshold

324

J. Opt. Soc. Am. A/Vol. 2, No. 2/February 1985

for discriminating whether a grating moves to the right or to
the left is the same as the contrast threshold for detecting thestimulus.1 2 This is consistent with the hypothesis thatmoving stimuli are detected by sensors that are direction selective and that explicitly assign a direction to the internal
representation of the stimulus. In other words, direction ispart of the label of the sensor.
This result is found only for images moving at moderate tohigh velocities (above about 1 deg/sec). At lower velocities,
the threshold for judging direction is much higher than thatfor simple detection. This is not due to the confounding effects of eye movements on slowly moving patterns, as it has
been demonstrated under stabilized conditions. 2 0 This discrepancy between results at low and high velocities may reflect
the operation of separate motion and static systems (seeSubsection 1.K).
I. Contrast Sensitivity to Moving Patterns
Robson 2l and others2 2,23 have measured human sensitivity
to flickering sinusoidal gratings with specific spatial andtemporal frequencies. The data form a surface as a functionof spatial and temporal frequency whose high-frequency
boundary is the product of separable spatial and temporalcontrast-sensitivity functions. 2' In other words, the upperbounds of sensitivity appear to be set by separate spatial andtemporal filtering processes. As noted in Subsection 1.K,
contrast thresholds for moving and flickering gratings arealways equal to within a factor of 2. Apart from this small
factor, then, this spatiotemporal contrast-sensitivity surface
determines sensitivity limits for moving gratings. Direct
measurements of sensitivity to moving gratings indicate that
this is so. 24 ,25
One consequence of the shape of the spatiotemporal contrast-sensitivity surface is that, as the image moves faster, thehighest spatial frequency that can be seen declines. As speed
increases, the temporal frequency (the product of speed andspatial frequency) increases most rapidly for the highest
spatial frequencies. We have already observed that motion
sensors must be selective for different bands of spatial frequency. The above result implies that low-spatial-frequency
motion sensors will respond to higher velocities than will
high-spatial-frequency motion sensors.
J. Apparent Motion
If a spatial target is presented at one location, then extinguished, and a brief time later presented again at a nearby
location, it may appear to move between the two points. A
summary of the extensive classical literature on this subject
is provided by Kolers. 26 The effect is more pronounced when
the sequence contains not just two but many presentations
along a path. 27 When the time and space intervals between
presentations are brief enough (as in film and video), the sequence of static presentations may be indistinguishable fromcontinuous motion. This limiting result is probably a consequence of the known spatial and temporal filtering actionsof the visual system and does not necessarily reveal anything
about the motion-sensing system per se. 29 ,30
Apparent motion is important for three reasons. First, it
is a good example of a stimulus without a well-defined opticalflow, which may yet give rise to unambiguous perceived motion. Second, it imposes another constraint on our model. It
too must respond to apparent-motion stimuli and should in-

A. B. Watson and A. J. Ahurnada, Jr.

dicate the same speed and direction as perceived by the observer. Third, it suggests a filter structure for our sensor.
The stimulus for apparent motion may be regarded as asampled version of a corresponding continuous motion. The
sampled stimulus differs from the continuous stimulus by theaddition of certain spatial and temporal frequencies. The
apparent similarity or identity of the two stimuli suggests thatthe additional frequencies have been filtered out. This leads
us to seek a filter structure for our motion sensor and to investigate what form it should have in the space-time frequency domain. 57 3 0 This investigation is carried out in
Section 2.
K. Motion and Static Systems
There is some evidence that the visual system contains two
separate systems to process and represent moving and static
imagery, respectively.' 7 The systems are thought to be distinct in their spatiotemporal sensitivity, the motion systembeing more responsive to rapidly moving patterns (low spatialand high temporal frequencies) and the static system more
responsive to slowly moving patterns (high spatial and low
temporal frequencies). In addition, the motion system isthought to be direction selective, in the sense that image
components moving in opposite directions are sensed byseparate mechanisms. The systems are also thought to differ
in the image representations that they provide to later stagesof visual processing: The motion system assigns velocity (orat least direction) values to components in an image; the static
system does not.
The evidence on this point is modest but favorable. As
noted above (see Subsection 1.G), contrast summation is direction selective only above about 1 deg/sec.1 2 Furthermore,
direction is judged correctly at threshold only above about 1
deg/sec (see Subsection 1.H).12 13,20 Discrimination of spatial
frequency at detection threshold appears to be much worse
when the temporal frequency is suited to the motion systemthan when it is suited to the static system. 3'
L. Inhibition between Directions
There is some evidence for inhibitory interactions between
the sensors for opposite directions. 3 2 33 Levinson and Sekuler32 showed that adaptation to the sum of leftward- andrightward-moving gratings produced less threshold elevation
for a rightward-moving grating than did adaptation to arightward-moving grating alone. In addition, it is difficult
to construct explanations of the motion aftereffect that do not
involve inhibition between the sensors for opposite directions.
M. Speed Discrimination
Some of the evidence cited indicates that motion sensors are
selective for spatial frequency and for direction. Anotherdimension in which they might be selective is speed. However, observers are quite poor at discriminating the speed ofmoving (or flickering) gratings at detection threshold. 31 34
Quite low speeds can be discriminated from quite high speeds,
but no finer performance is possible. This suggests that thesensors are not particularly selective for speed or that thesensors that detect moving images do not explicitly assign aspeed value. Above threshold, however, the system is exquisitely sensitive to variations in speed, reliably detecting
variations on the order of 5%.14 This discrepancy between

Vol. 2, No. 2/February 1985/J. Opt. Soc. Am. A

A. B. Watson and A. J. Ahumada, Jr.

325

threshold and suprathreshold discrimination is a common
feature of human perception (see, for example, the comparable
case in discrimination of spatial frequency 31 ' 35) and may be
explained by a two-stage model. The second stage, whichbecomes effective only above threshold, combines the responses of first-stage mechanisms and provides finer performance. Our motion sensor will take this two-stage form.

where -2 indicates the 2D Fourier transform. Translationof the image can be represented as a coordinate transformation in which the x coordinate increases linearly with time.
Let r be the speed of horizontal motion. If we write a' = ()for the transformed coordinates, then

2. FREQUENCY COMPOSITION OF MOVINGIMAGES

From the general expression for the transform following anaffine coordinate transformation (any combination of scaling,rotation, and translation), 3 7 we find that

Section 1 described some properties of human motion perception that will guide the construction of our motion sensor.
One of those properties was that motion analysis appears to
be done in parallel within a number of separate bands ofspatial frequency and that the elementary features into which
images are decomposed are patches of oriented sinusoid. This
leads us to consider how the frequencies that make up an
image behave as the image is moved. 5 2830 ,36 We restrict ourattention to one particular case: that of an otherwise-unchanging image undergoing translation at a constant velocity.
This is one of the rare cases in which an unambiguous image
velocity exists. It should be noted that other, more-complex
motions can be constructed by piecing together spatial andtemporal segments of this simple kind.
A. Contrast Distribution of a Moving Image
An arbitrary monochromatic space-time image can be represented by a function c(x, y, t) defined over some interval,
which specifies the contrast at each point x, y and time t.
Contrast is defined as the luminance divided by the averageluminance over the interval. Although not strictly necessary,it is convenient to begin this discussion with an image that is
static, that is,co(x, y, t) = co(x, y, 0)

for all t.

(1)Define the image velocity as the vector r with horizontal andvertical speed components r and r, or, in polar terms, a speedr and direction 0, where r = r cos and r = r sin 0. If thepreviously static image translates at constant velocity r, thedistribution becomes
cr(x, y, t) = c(x - rt, y - ryt, t).

1. Two-Dimensional Case
For simplicity, we develop the two-dimensional (2D) case ofc(x, t). This would be adequate to describe images without
variation in the vertical dimension, for example, a vertical lineor grating. We adopt the following vector notation:b = (j),(3)

where u and w are the spatial- and temporal-frequency variables corresponding to x and t, respectively. Then a function
and its 2D Fourier transform can be written asc(a) -2(b),It) = Aa,iFr] _A=(5)Oa') -2,ff[(A-1)TbI,(6)

where the superscript [ IT indicates the matrix transpose
operation. Since(A-1)T=[l°,(7)

we end up withc(x - rt, t) -2(u,w + ru).

(8)

Graphically this means that moving the image shears its
spectrum in the w dimension. Spatial frequencies are notchanged, but all temporal frequencies are shifted by minus
the product of the speed and the spatial frequency, -ru. This
result is easiest to picture for an image that was static in time
before the introduction of motion, as in Fig. 1. The spectrumof the static image lies entirely along the u axis (all signal
energy is at 0 Hz). When the image moves, the spectrum issheared as described above, so that the result lies along a line
through the origin with a slope of -r.
2. Three-Dimensional Case
The preceding can be generalized to three dimensions. If thevelocity is r = (r, ry), the moving image and its transform arethenc(x - rt, y - ryt, t) *3 c(u, v, w + rxu + ryv).

(9)

W

(2)

B. Fourier Transform of a Moving Image
Given an arbitrary space-time image c(x, y, t), and its Fouriertransform (u, v, w), we seek a general expression for thetransform of this image as it undergoes translation at a constant velocity r.

a = (x) ,

a'=

(4)

U0
U
0

Fig. 1. The effect of motion on the Fourier transform of a 2D
space-time image. The effect is shown for a single representative
component of spatial frequency uo. The open circles show the location of the components of the static image. The dotted circles show
the locations when the image moves with speed r. Each transform
point is sheared in the w dimension by an amount -ru, as indicatedby the arrows.

326

J. Opt. Soc. Am. A/Vol. 2, No. 2/February 1985

A. B. Watson and A. J. Ahumada, Jr.

Each temporal frequency is shifted by an amount -(ru +ryv). If the original image was static (w = 0), then the new
temporal frequency will be equal to this quantity. Since itwill play an important role in subsequent developments, wenote that this quantity is the dot product of the two-dimensional spatial frequency vector f = (u, v) and the image-velocity vector r = (rx, ry). Three possible expressions for the
temporal frequency of each component of the spectrum of animage in motion are thenw = -r- f = -(r.u + ryv) = -rf cos( - a),(10)

where 0 is the direction of image motion and a is the orientation of the spatial-frequency component. This quantity canbe seen to be the product of the spatial frequency f and thecomponent of the velocity in the direction equal to the orientation of the grating.3 8 Note that the two-dimensional
result described earlier is a degenerate case of this generaloutcome.
Geometrically, image motion changes the static-imagetransform, which lies in the u, u plane, into a spectrum thatlies in an oblique plane whose intersection with the u axis hasa slope of -ry and whose intersection with the v axis has aslope of -rx (the dihedral angle of the plane will be tan- r).
To illustrate, consider a stationary sinusoidal grating of frequency f and orientation a. As illustrated in Fig. 2, its
transform is a pair of three-dimensional (3D) impulses at uo,
vo, 0 and -uo, -vo, 0, where u0 = f cos a and vo = f sin a. If
the grating moves in direction 0 at speed r, applying Eq. (10)shows that the spatial frequencies will not change, but thetemporal frequencies will shift to minus and plus rf cos(O a), respectively. These two points lie at opposite ends of aline through the origin.
In summary, the spectrum of a stationary image lies in thea, v plane. When the image moves, the transform shears into
an oblique plane through the origin. The orientation of thisplane indicates the speed and direction of motion.
The preceding discussion shows that it is possible to asso-

Vw

Fig. 2. The effect of motion on the transform of a 3D space-time
image. The effect is shown both for a plane representing the full
spectrum and for a single representative component of spatial frequency f and orientation a. The solid plane and filled circles show
the location of the spectrum of the static image defined by w = 0.
Motion at velocity r shears the spectrum into the plane w = r b, asshown by the dashed-dotted plane and dotted circles. The arrows
indicate the displacement of a single spatial-frequency component.

Fi.

.example

.:

o

dc .u

.i

. ami
. .

T.in

t

::
~~~~~~.::........
:~~~~~~~~.
. . . . . ..... . . . .

-.

-

.

.. ..

~~~~~~~*.
. . . ......
. . .. ....
. . ... . .

Fig. 3. An example of direction ambiguity. The motion of thecontour seen through the aperture is consistent with any of the velocities a or b or c (after Marr and Ullman). Note that all possible
velocities have the same velocity component orthogonal to the contour.39
ciate energy in particular regions in spatiotemporal frequency
space (for example, the one pictured by the dotted circles inFig. 2), with particular image-velocity components. By filtering specific regions, it is therefore possible to detect
image-velocity components of particular values. This observation forms the basis for the development of our scalarmotion sensor. However, as noted below, a single imagevelocity component is ambiguous. The ambiguity must be
resolved by combining several measurements of different
components of the image velocity.
C. Direction Ambiguity
When a straight contour moves behind an aperture (Fig. 3),
its direction is ambiguous. The stimulus is consistent witha range of velocities whose directions cover 180 deg. This has
been called the apertureproblem.3 9 In fact, the aperture isnot critical, for the same ambiguity will result from a patternof infinite extent, provided that all its frequency components
have the same orientation. The aperture plays its role bylimiting the visible portion of a pattern to a region within
which only a single orientation is present.
This ambiguity is easily seen in the frequency-space diagram of Fig. 2. The plane, which contains spatial-frequency
components at all orientations, uniquely defines an image
velocity of r, 0. However, the solid circles, which represent
a single component with just one orientation, lie within many
possible planes and are thus consistent with many possible
velocities. Similarly, a moving image whose spatial
frequencies all have the same orientation has a spectrum thatlies along a straight line through the origin. This line is contained within an infinite number of planes and is thus consistent with an infinite number of velocities. Specifically, if
the orientation of the image components is a, and the speed
in direction a is r, then the transform (and image) corresponds to any velocity ro, 0 such thatro = r cos(G - a).

(11)This ambiguity has traditionally been a problem for motion
sensors that operate on individual oriented components in theimage. The problem of how these ambiguous oriented components are resolved into a single unambiguous velocity estimate is dealt with in Section 4.

Vol. 2, No. 2/February 1985/J. Opt. Soc. Am. A

A. B. Watson and A. J. Ahumada, Jr.

D. Summary
In Section 1, we observed that moving images are sensed interms of their oriented 2D spatial-frequency components.
Here, we have shown that the spectrum of a moving 2D spatial-frequency component occupies two regions at opposite
ends of a line through the origin. A sensor designed to filter
such antisymmetric regions in frequency space should
therefore sense these moving components. This sensor isconstructed in Section 3. However, as noted in Subsection
2.C, individual oriented components are ambiguous as to direction, in that each indicates only one scalar component of
the image velocity. Therefore additional steps must be taken
to remove this ambiguity. We take up this point in Section
4.

3.

THE SCALAR MOTION SENSOR

We construct our model in two stages. The first stage is anarray of linear sensors that are selective for 2D location, 2D
spatial frequency, and direction. Because, at any momentin time, each produces only a single number rather than thevector required to specify velocity, we call it a scalar motion

327

sensor. The second stage, discussed in Section 4, is called thevector motion sensor.
The scalar sensor can be described in two ways. One is amathematical derivation; the other is a physical or physiological description, without reference to mathematical detail.
We begin with the mathematical treatment.
The overall mathematical structure of the scalar sensor is
illustrated in Fig. 4. It is an assembly of spatial and temporal
filters each of which has simple properties. In assemblingthese components, two points should be remembered:(1) When several filters are in series, their order is irrelevant. This is formally expressed in the commutative property of convolution, f * g = g * f. Consequently, the order inwhich we introduce components is not necessarily the orderin which their physical analogs are connected.
(2) The filter that we are constructing operates on thethree dimensions x, y, t or in frequency space u, v, w. A
separable filter is one in which the impulse response is the
product of one-dimensional (ID) components. Separability
permits certain simplifications; for example, the 2D or 3D
transform of a separable function is the product of the separate ID transforms, and the 2D or 3D convolution of separable
functions is the product of their separate D convolutions.
For these reasons, we will deal separately with temporal,
horizontal, and vertical filters, which should be understoodas the ID components of 3D filters. For example, f (t) is thetime component of a 3D impulse response f (t)3(x)6(y).

TEMPORAL FILTERTEMPORAL DELAY

A. Basic Temporal Filter
We introduce a temporal filter with impulse response f (t) andtransfer function 7(w). We have tentatively identified thisfunction with the temporal sensitivity of the human observer
to relatively low spatial frequencies. 2 1-23 ,40 ,41 A useful analytic approximation to these data is provided by the impulse
responsef(t) = W1l(t) - th(0),

(12)

where
SPATIAL FILTER
fi(t) =HILBERT SPATIALFI LTER

u(t)

rini-

1)!

(t/r)ni-le-t/ri(13)

and u(t) is the unit step function. The function f is an nstage low-pass filter, essentially as used by Fourtes andHodgkin.4 2 This analytic form of f(t) has been used elsewhere
to model temporal contrast-sensitivity data. 41 43 The filtertransfer function is given by the Fourier transform of theimpulse response,7(w) = (Ih(w) - V12(W)I,(14)

Jj(w) = (i27rw i + )-

(15)

where
HILBERT TEMPORALF LTER

Fig. 4. Mathematical structure of the scalar motion sensor.

We have chosen parameters (t = 0.9, 1 = 0.004, 2 = 0.0053,
n = 9, n2 = 10) for this filter that fit the data of Robson.2 'Impulse response, amplitude response, and phase responseof the filter are shown in Fig. 5. The operation of the sensor
is not critically dependent on the particular version of f(t)selected. An alternative choice might be the temporal impulse responses of individual visual cells, but these appear tobe quite varied.

328

A. B. Watson and A. J. Ahumada, Jr.

J. Opt. Soc. Am. A/Vol. 2, No. 2/February 1985

C
0U

0

0)6

~0,

~0
0
L

D u
4-I

CD
:O

%<EI

0 I

1.

.

I

.

1

5 10

50

<0
cm

U-)
7

0

40

Time

80

120 160(msec)

0,_,-

1

-

I

Frequency

(Hz)

5 10

50

.I

Frequency(Hz)

Fig. 5. A, Impulse response; B, amplitude response; C, phase response of the basic temporal filter. The symbols in the center panel are contrast-sensitivity measurements made by Robson 2 l for a 0.5-cycle/deg grating.

a

1

b

1

Fig. 6. Spatial impulse responses of a, main and b, quadrature paths. The spatial impulse response of theebasic spatial filter is equivalent
to that of the main path, a. The gray background indicates a value of 0.

B. Basic Spatial Filter
We next introduce a spatial filter with impulse response g(x,y) that is separable into horizontal and vertical components
g(x, y) = a(x)b(y),(16)2a(x) = exp[ -(x/X) ]cos(2rux),(17)b(y) = exp[-(y/X)2].

(18)

This impulse response is pictured in Fig. 6. It is the productof a cosine and a Gaussian in the horizontal dimension and isa Gaussian in the vertical dimension. The frequency of thecosine is us, and the Gaussians both have spreads of X. Thisfunction, sometimes called a 2D Gabor function, is a good
approximation to the receptive-field weighting function of
so-called simple visual cortical cells in cat and monkey 44 andpresumably in the human. It has also been used as a modelof the basic spatial operator in theories of human spatial vision. 45 ' 46

The transform isg(u, v) = a(u)5(v),(19)d(u) = \WX/2(expl-[7rX(u -U)]21+ expl-[rX(u + U,)] 2 )),2

6(v) = /-X exp[-(7rlv ) ].

(20)

(21)

C. Size of the Spatial Sensor
The parameter Xgoverns the size of the Gaussian componentof the spatial impulse response. Here, we set it equal to a
constant p times the period of the sinusoid,
X = P/f.

(22)

Thus the size of the sensor scales inversely with its spatial
frequency. This has the consequence of fixing the log-frequency bandwidth of the sensor. We assume that p =3N/UiH2hr = 0.795, in which case this fixed log bandwidth isone octave. This particular bandwidth is chosen to be consistent with available psychophysical and electrophysiologicalevidence.
With the introduction of the basic spatial and temporal
filters we have traversed the first two boxes in Fig. 4. The
resulting transfer function will be the product of the temporaland spatial transfer functions. Since each of these has twoVol. 2, No. 2/February 1985/J. Opt. Soc. Am. A

A. B. Watson and A. J. Ahumada, Jr.

329

lobes, one on either side of the frequency origin, their productwill have four lobes rather than the two antisymmetric lobes
required by a direction-selective sensor (see Fig. 1). Our next
step then is to null two of the four lobes of the filter transfer
function and thus make the sensor direction selective. We
do this by means of a Hilbert filter.
D. Hilbert Transform
We digress to introduce the Hilbert transform, which is dealt
with in most standard treatments of linear systems theory but
is rarely seen in the vision literature. The Hilbert transformof a function g(x) is given by h(x) * g(x), where

_
/

D..
v.

.

..
**
.

.

.

a-

(23)

h(x) = -1/rx.

The Hilbert transform is a linear filter with an impulse response that is an inverted hyperbola. The transfer functionof the filter is the Fourier transform of h(t):h(u) = i sgn(u).

(24)

This filter, pictured in Fig. 7, has the remarkable property of

0

50

ISO

Time

150

200

250

(msec)

Fig. 8. Temporal impulse responses of main (solid line) and quadrature (dotted line) paths of the scalar motion sensor.
unit gain at all frequencies combined with a constant phaselag of.,ir/2 for positive frequencies and -r/2 for negative
frequencies. It converts odd functions into evens, and evens
into odds. Two functions that are Hilbert transforms of eachother are said to form a quadrature pair.
I__ ___ ___ ___ ___Time Delay
The next element in the sensor is a time delay of r. This canbe represented by convolution by a delayed impulse 8(t - r).
The delay has a transfer function of exp(-i27rwr). This delay
is introduced to ensure that the introduction of the Hilbert
filter, described below, does not result in a noncausal temporal
impulse response.

___E.

F. Main and Quadrature Paths
At this point, the signal path branches into what we will callmain and quadrature paths. The quadrature path is subject
to a Hilbert transform in each of the dimensions of t and x.

a

I

I

.....................

b
Fig. 7. The Hilbert filter. a, Impulse response. b, Transfer function. The dotted line indicates an imaginary value.

G. Hilbert Temporal Filter
In the quadrature path, the signal passes through a temporal
Hilbert filter, so that the temporal impulse responses of mainand quadrature paths are now f(t - r) and f(t - r) * h(t),respectively. These are illustrated in Fig. 8. The Hilbert
impulse response, when viewed as a function of time, is neither
causal nor physically realizable. However, we need only ensure that the result of the transform (the dashed curve in Fig.
8) is causal and realizable, which is easily done through suitable choice of the delay r. The Hilbert impulse responsecontains infinite values, but we make use of it in the frequency
domain, where it presents no difficulties of calculation.
H. Hilbert Spatial Filter
The quadrature path is now subjected to another Hilberttransform, this time in the horizontal spatial domain. The
resulting spatial impulse response in the quadrature path is[h(x) * a(x)]b(y)

(25)

330

J. Opt. Soc. Am. A/Vol. 2, No. 2/February 1985

A. B. Watson and A. J. Ahumada, Jr.
mq(u, v, w) = -fm.(u, v, w)sign(u)sign(w).

I,,,,i:;;t' 000 t0000 ft00 fiV00000; Si ......... ',.004 ',;
,!0

I,,;

The final step is to combine the two paths. For a sensor ofrightward motion, we add the two paths,

S L+; ,-.,

fftr(U, v, w) = mm + mhq = a(u)6(v)J(w)

X exp(-i2rwr)[1 - sign(u)sign(w)].

I've::
f:

Cty 0

fD

i :4SD::000S00

lA000,i;Aid;fiV000

And

All f

Z -Ei;

l,'''t00!
+,;00
.--I;
t:;

S:f

. .

'

(29)

The modulus of this transfer function (the amplitude response), and the corresponding impulse response, are shownin Fig. 9. Note that the amplitude response occupies only thesecond and fourth quadrants of the u, w space. Thus the effect of adding the quadrature path is to null two of the four
lobes of the transfer function, so that it occupies only two diagonally opposite quadrants in frequency space. Recall thatthese are the quadrants occupied by an image moving to theright (Fig. 1). An image moving to the left will occupy the first
and third quadrants and will therefore produce no response.
The sensor is therefore selective for motion to the right.

f:;0:f

:0StA
An.f
|

(28)

S

; f aVlf'tS;S-aJ. Generalization to Other Directions
Each scalar sensor has a particular spatial frequency fs anddirection s. These numbers can be compactly represented
by a vector s = ( 8 , us), where us = s cos As and v = fs sin 0.
We will call this the directed frequency of the sensor. So far
we have developed the case of a sensor for horizontal motion
to the right, that is, = 0. The sensor for an arbitrary direction 0t is obtained by rotating the x, y space by -0. Thismeans that we are using a new spatial weighting function withorientation O. In the frequency domain, this has the effectof rotating the transform by 0,, which we accomplish by thecoordinate transformationu' = u cos As + v sin 0,

u' =-u sin+ v cos .

(30)

2

Let G = rX /2. This is the overall gain of the motion sensor.
Then the transfer function of a sensor of directed frequencybFig. 9. a, Impulse response and b, amplitude response of a scalarsensor for leftward motion (direction = 0). In a, the axes are x (horizontal) and t (vertical). In b, they are u and w.
with a Fourier transformi sign(u)d(u)i(v)(26)

Equation (20) shows that (u) is two Gaussians at u and - 8with spreads of 1/(7rX). If, as in the present case, X > 2/(7ru,)
then each Gaussian will be almost entirely on one side of theorigin, in which case the transform will be the sum of twocomplex Gaussians, one positive at us, the other negative at-us. Taling the inverse transform shows that the resulting
impulse response in the quadrature path is simply (approximately) equal to minus a sinusoid multiplied by a Gaussian.
Thus the spatial impulse responses of the main and quadrature paths are even and odd (cosine and sine phase) Gaborfunctions. They are shown in Fig. 6.
I. Sensor Transfer Function
We have applied Hilbert filters of t and x to the quadrature
path; hence its transfer function 6t q will differ from that of themain path mm by multiplication by the transfer functions ofthese two Hilbert filters,lm(u, v, w) = a(u)6(v)J(w)exp(-i27rwr),

(27)

is

rhs(f, w) = Gfexp[-(r1s - fl) 2] + exp[-(rXgs + )2]1
X J(w)exp(-i2wrrw)[1 - sgn(s - f)sgn(w)].

(31)

Note that the first term is a pair of Gaussians at s and -s.
The impulse response and the amplitude response of thescalar sensor for one direction are shown in Fig. 10.
K. Cardinal Form for a Direction-Selective Sensor
It is evident that the essential feature of our direction-selective
sensor is that it responds in just two antisymmetric quadrants
of frequency space. In the preceding sections we have developed a particular example of such a sensor, but it is worth
considering which of its features are essential to meet theabove criterion. A canonical form for our linear direction
selective sensor (in the x direction) is
a(x)b(y)c(t) * [(x, y, t) + (y)h(x)h(t)],(32)

where a, b, and c are arbitrary separable functions of space
and time. In other words, the filter consists of the productof separable functions, plus its own Hilbert transform in x andt. The canonical forms for other directions are obtained
through suitable coordinate rotations. Thus it is the structureof form (32) that gives the direction selectivity; the functional
forms of a, b, and c have been chosen to fit other aspects ofhuman motion perception.

Vol. 2, No. 2/February 1985/J. Opt. Soc. Am. A

A. B. Watson and A. J. Ahumada, Jr.

L. Simple Description of the Scalar Motion Sensor
As promised at the start of Section 3, we can now translate thepreceding discussion into a simple physiological descriptionof the sensor. We imagine a pair of visual neurons, withspatial receptive fields as pictured in Fig. 9. The functionsin this figure now describe the influence of light upon each cell
as a function of the distance of the point of light from thecenter of the receptive field. Second, we interpret the two
curves in Fig. 8 as the temporal impulse responses of the twocells. For example, the solid curve in Fig. 8 is the time response of the even cell (Fig. 9a) to a brief pulse input, whereas
the dashed curve is the similar response for the odd cell. We
then imagine a third cell that simply adds the responses of the
odd and the even cells. This third cell will be direction selective and is a physical embodiment of our scalar motion
sensor.
M. Response of the Scalar Motion Sensor
The response of the sensor to an arbitrary jnput is given directly by convolutionr(x, y, t) = c(x, y, t) * m(x, y, t)(33)

or by way of the convolution theorem
r(x, y, t) -3 d(u, v, w)fi(u, v, w).

(34)

The mathematical development of the sensor as a filter results
in a response that is itself a 3D function of space and time.
Yet when speaking about a sensor as the theoretical analog
of a single visual cell we expect its response to be a functionof time only. This apparent contradiction is resolved byviewing r(x, y, t) as specifying the response at time t of thesensor centered at x, y.
What is the form of this temporal response? The sensor
removes all spatial frequencies save those in the neighborhoodof s. As noted in Subsection 2.B.1, the input at s has a temporal frequency of s *r. Since the sensor is linear, the output

fff
V

t

331

will have the same frequency as the input. Thus the temporal
response of the sensor will approximate a sinusoid with atemporal frequency of(35)

ws = s r.

Comparing this with Eq. (10), we note the following useful
result: The temporal frequency of the sensor response approximates the component of the image velocity in the sensor
direction, multiplied by the spatialfrequency of the sensor.
We can think of the temporal frequency as coding one component of the image velocity.
N. Selectivity of the Scalar Motion Sensor
Examination of the transfer function of the motion sensor
reveals its selectivity for direction, orientation, spatial frequency, temporal frequency, and the like. A simple way ofshowing this is by considering the response to sinusoidal
gratings of various spatial frequencies, orientations, directionsand velocities. Since the system is linear, the temporal response to this stimulus will be sinusoidal with the same temporal frequency as the input. It will have an amplitude thatcan be read directly from the amplitude response given in Eq.
(31). For example, consider a 2D sinusoid of spatial frequency
f and velocity r. From Eq. (31) it is evident that the response
will be a sinusoid of amplitude
G exp[-(7rXl s - fl) 2jI(I sI Ifl )l[1 - sign(s r)].

(36)

This expression can now be used to illustrate the sensitivityof the sensor to spatial frequency and direction.
1. Spatial Frequency
If grating direction matches that of the sensor ( = Os), thenthe sensor response amplitude is given by
G exp-[fr>( - f)] 2JI,(rf)I.

(37)

When the temporal frequency w = rf is fixed, this amplitude

7w

Fig. 10. a, Impulse response and b amplitude response of a scalar sensor for motion in direction = 3240. In a, frames show successive timesamples of 12.5 msec. In b, frames show successive temporal frequency samples of 5 Hz, with the origin in frame 9.

J. Opt. Soc. Am. A/Vol. 2, No. 2/February 1985

332.

A. B. Watson and A. J. Ahumada, Jr.

0
43
4)
-4

EL

-1 0

-0. 5

0.0

0. 5

-1. 0

1.0

Spatial Frecquency (cycles/degreee)

-0. 5

0. 5

0. 0

Spatial Frequency(cycles/Jegree)abFig. 11. Sensor-response amplitude as a function of the spatial frequency of a moving sinusoidal grating. a, Constant temporal frequency.
b, Constant velocity. The shape of the curve is determined by the product of speed and sensor spatial frequency. The three curves are for valuesof 1 (solid), 16 (dotted), and 32 Hz (dashed).
2.

Direction

The sensitivity of the sensor to the direction (and orientation)of a grating input can be determined by setting the spatial
frequency of the grating to that of the sensor (f = fs). Thenthe sensor response amplitude as a function of grating direction will be-oG exp(-r 2 s2 [ 2 + f2- 2ff cos(0 - Os)])1f(rf)l.

D

(38)

The variation with orientation is captured in the exponential
term, which simplifies to

IL\

E

exp

SI

I

I

-50
0
50
Direction (degrees)Fig. 12. Normalized response amplitude of the scalar motion sensor
as a function of the direction of a moving sinusoid [Eq. (38)] with p[27r sin (-

)]}

(39)

This function is drawn in Fig. 12. Making use of the approximation sin(x) x for small x, we see this is approximately a Gaussian.function of the difference in direction between the sensor and the grating:exp{-[7rp(0 - A.)]21-(40)

This function is not visually discriminable from Eq. (38)shown in Fig. 12. If p = 0.795, as has been assumed here, thenthe sensor has an direction bandwidth of about 38°.

= 0.795.

varies as a Gaussian function of the distance between grating
and sensor frequencies. When X= 0.795/f, as assumed here,
the Gaussian has a width of one octave (2f/3) at half height.
If the velocity is held constant, then 1(rf)l will also vary,
though not as rapidly as the Gaussian. These two cases are
slices through the 3D amplitude response shown in Fig. 10b.

They are illustrated in Fig. 11. The figure shows that at very
low or very high speeds, the spatial-frequency band of thesensor can be shifted somewhat from its nominal center frequency.

4.

VECTOR MOTION SENSOR

A number of the results cited in Section 1, particularly thosethat deal with events near detection threshold, suggest thateach direction is served by an independent sensor, such as thescalar sensor developed in Section 3. Other results, however,particularly those that involve estimating the apparent speed
and direction of a superthreshold pattern, indicate the cooperative action of a number of sensors tuned to different directions.
Furthermore, we have seen that the output produced by an

A. B. Watson and A. J. Ahumada, Jr.

individual scalar sensor is ambiguous with regard to the speed
and direction of the image. Here, we show that this ambiguity
can be resolved by combining the responses of a number of
sensors. This combination is done by the second stage of themodel, which we call the vector motion sensor. Before webuild the vector sensor, however, we describe how the individual scalar sensors are distributed over space, spatial frequency, and orientation.
A. Distribution of Sensors
We have developed a model of a single sensor tuned for aparticular location x8, y 8, spatial frequency f8, and direction
A. The full model contains many sensors, replicated overspace, frequency, and orientation according to the following
rules, which are simplified from an earlier model of spatial
sensing. 46
1. Spatial Frequency
As noted in Section 3, we have assumed that each sensor has
a half-amplitude spatial-frequency bandwidth of one octave.
This constrains the choice of the number of separate frequency bands to be analyzed, since too many will be redundant, whereas too few will result in a loss of information. We
adopt an interval of one octave between bands. To cover therange of visible frequencies, we assume a set of eight sensor
center frequencies of 1/4, 1/2, 1, 2, 4, 8, 16, and 32 cycles/deg.
Further investigation may reveal whether all these bands arenecessary or whether, for example, motion analysis is confined
to lower spatial frequencies.

2. Direction
As noted in Section 3, the orientation bandwidth of eachsensor is about 380. To cover the full range of directions atintervals of about one bandwidth, we therefore assume 10
different possible sensor directions, moving in steps of 36°
from 00. These 10 directions can be constructed from the fiveorientations and two phases assumed in the spatial model
described by Watson. 4 6
3. Spatial Sampling
Loss of information through aliasing will be prevented if thesampling density of the sensors (the inverse of the distance
between adjacent sensors) is at least twice the full-amplitude
bandwidth of the sensor. We approximate this condition byuse of a sampling density of four times the center frequencyof the sensor (six times the half-amplitude bandwidth). With
this density, the sampling artifact and the original spectrum
overlap at a negligible amplitude. The actual sampling
density required to account for human performance is asubject for further study.
Since the sampling density is proportional to frequency,
there will be many more sensors at high spatial frequencies
than at low. For example, in one square degree of visual field
there will be one sensor of the lowest frequency (1/4 cycle/deg)
but 16,384 sensors of the highest frequency (32 cycles/deg).
There are several aspects of the way in which sensors arelikely to be arranged in the human visual system that we have
not yet tried to include in our model. Most prominent is theway in which sensors may change in size with distance fromthe fovea. Second is the approximate hexagonal packing of
sensors across the visual field. Both features are included in
the model on which the spatial features of this model are

Vol. 2, No. 2/February 1985/J. Opt. Soc. Am. A

333

based, but for simplicity we assume here a homogeneoussquare sampling array.
B. Direction GroupAs noted, each scalar sensor provides ambiguous information
about the image velocity, but this ambiguity can be resolved'
by combining scalar sensor responses. This combination isdone within each set of 10 scalar sensors at the same locationx, ys and spatial frequency f, that differ in direction 0t. We
call each such collection a directiongroup. Consider an image
moving over this group. If the image contains many different
orientations at frequency f, then many members of the groupwill respond. Since each corresponds to a different direction,
no single one can indicate the actual direction in which theimage moves. How do we deduce this actual direction fromthe responses of the group?
The response of any one scalar sensor does not indicate thevelocity because of the direction ambiguity problem discussed
earlier. Each scalar sensor conveys only one component of
the image-velocity vector-the component in the sensor direction. But the responses of several sensors in the directiongroup provide several linearly independent components of thevelocity vector. Therefore the full velocity vector can be
determined from several (ideally, just two) scalar-sensor responses. This conversion of the scalar-sensor responses ofthe direction group to a single estimate of image velocity isdone by the vector sensor.
Recall that the image-velocity components are coded in thetemporal frequency of the scalar-sensor responses. If theimage moves with velocity r, then the temporal frequency ina sensor of frequency s will be w8 = r *s. Converting this topolar terms, we see that the pattern of temporal frequencies
among the sensors within the direction group is(41)

w = rf8 cos(O - 00

where (r, 0) are the speed and direction of the image and (fA,A) are the spatial frequency and direction of the sensor. Thisfunction describing the temporal frequency of the responseof the sensor is a cosine with a phase equal to the actual direction of image motion and with an amplitude equal to the
product of sensor frequency and image speed. Only half ofthe cosine can be estimated, since for half of the possible directions the sensor will be silent (the sensor responds only if
its direction is within 90° of the direction of image motion).

SENSORSFREQUENCY
METERSMETERRESPONSESVELOCITY
ESTIMATORIMAGE_

Fig. 13.

The structure of the vector motion sensor.

334

J. Opt. Soc. Am. A/Vol. 2, No. 2/February 1985

There are a number of possible ways to extract the image velocity from this pattern of responses. The one that we havepursued is first to measure the temporal frequency of eachscalar-sensor response (by means of a frequency meter) andthen to fit Eq. (41) to the resulting quantities. This sequenceis illustrated in a process diagram of the vector motion sensor
in Fig. 13.
C. Inhibition between Opposite Directions
When an image moves past a direction group, it will excite halfof the sensors in the group. This is because each sensor has
a direction range of 180. However, noise may generate responses in the remaining half of the sensors. We know,
however, that only one of the pair of sensors tuned for opposite
directions should respond to an image in motion. Thus we
can null the noise responses by making each sensor competewith its opposite number. The larger of the two responses ispreserved; the smaller is discarded. In fact, to make things
simpler, we replace the smaller with the negative of the larger.
The result is that the temporal frequency as a function of
sensor direction is now a complete cycle of a cosine, rather
than just a half cycle.
D. Summary
The result of all of the preceding operations will be a set ofeight sampled vector fields, one per scale or spatial-frequency
band. Each sample will be a vector indicating the velocity(speed and direction) of the image Gabor component at thatlocation. The density of samples in each field will be proportional to the square of the spatial frequency. In Section
5, we describe some specific details of how these vector fields
may be computed. Although we do not attempt at this timeto provide a specific linking hypothesis between these vectors
and perception, some common-sense predictions are possible.
For example, if the vectors at all scales and locations agree,
we predict that the image will appear to move with the specified velocity.
5.

A. B. Watson and A. J. Ahumada, Jr.

C. Scale
The input is analyzed in parallel by eight sets of sensors, each
set selective for a different spatial-frequency band. The action of each set is identical to that of every other except for achange of spatial scale. This scale invariance allows us todescribe the action of the sensors at just one scale k.
Consider an input digitized to a movie of size W X H X L.
In the following discussion, we assume for simplicity that W= H. The digital movie contains spatial frequencies up to the
Nyquist limit of 2-1 W cycles/width. The highest-sensorfrequency band considered will be centered at 2-2 W cycles/
width. For convenience, we assign a scale number of 0 to thishighest-frequency set of sensors applied to a given input.
Since the sensors change spatial frequency in octave steps, thescale-k sensors will have a spatial-frequency passband centered at 2 -(+2) W cycles/width.
Scaled Inputs The band-limited character of the sensors
at each scale also permits an important computational economy. As noted, the input contains spatial frequencies up to
W 2-1. The scale-0 sensor has a center frequency of W 2-2
and an effective passband cutoff one octave about this point,that is, W 2-1, equal to the cutoff frequency of the input [infact, this is a very conservative figure, as the sensor frequency
response has fallen by about 99.8% (54 dB) at this frequency].
At scale 1, the cutoff frequency of the sensor is reduced by afactor of 2 to W 2-2. Frequencies above this point may beremoved from the input without altering the response. Afterthese frequencies are removed, the number of samples required to define the input may be reduced by a factor of 2 ineach spatial dimension. We define the shrink operator (denoted Sh) as one that removes the frequencies above one halfof the Nyquist limit and then reduces the number of samples
in each spatial dimension by a factor of 2. Our shrink algorithm is described in Fig. 1 4 .4749 We define Ck, the scale-k
Co0

Co0

IMPLEMENTATION OF THE MODEL

A. General
The current implementation is written in the language c under
the UNIX operating system. It currently resides on a SUN
computer (an MC68010-based virtual-memory microcomputer with dedicated graphics display). The software isportable to other UNIX installations and will be described ina subsequent report.
B. Terminology
The input to the model is a continuous distribution of luminance or of contrast over the three dimensions of x, y, and t.
A discrete representation of this input can be constructed by
Nyquist sampling, resulting in what we will call a movie, a 3D
array of values with dimensions W (width), H (height), andL (length). The values in the array may be either real orcomplex. A movie may be used to represent a sequence ofimages, its 3D Fourier transform, or any data set that fits
within this structure. When used to describe a discrete
moving image, the natural units of the movie are pixels andframes (or widths and lengths). When used to describe thetransform of a discrete moving image, convenient units arecycles/width and cycles/length.

fft

C,

Fig. 14. The shrink algorithm. The original image of width W is0denoted C0 . Application of a fft results in C
o. It contains frequencies
up to 2-1 W. A square core, which contains frequencies up to 2-2 W,
is inverse transformed to yield a movie C1 that is half as large in eachspatial dimension. The procedure can be repeated to yield C2, andso on. The algorithm can be generalized to other size ratios and to

the time dimension. This procedure is similar to other pyramid
schemes47 -49 but has the advantage of precisely band limiting thesignal before subsampling.

Vol. 2, No. 2/February 1985/J. Opt. Soc. Am. A

A. B. Watson and A. J. Ahumada, Jr.

335

quency domain, multiplying, and inverse transforming. Eachspatial sample in the result is the time waveform of the response of the sensor at one location. This sequence of operations for scale k and direction I is pictured in Fig. 15.

47 fftk

1. Sensor Impulse Response
The impulse response of a sensor at scale 0 and direction I isdefined by a real movie M0,1 of the same width, height, andlength as the input. The center spatial frequency of thesensor is W 2-2 cycles/width, where W is the input width inpixels. For example, if the input is a movie of width 32, height
32, and length 16, then the sensor impulse response at scale
0 will be the same size and will have a center frequency of 8
cycles/width. Values of the impulse response are sampledfrom Eq. (31). Figure 11a shows an example of an impulse
response.

ft(4-

2. Sensor Transfer Function
To obtain the transfer function of the sensors at scale 0, thescale-0 impulse response is transformed:

4fft-I

fMft
Mo, 1,(43)-s

Fig. 15. Computation of Ral, the sensor response at scale k, direction
1. Shaded objects are Fourier transforms.

representation of the input, to be the result of applying theshrink operator to Ck-1, and we write this as
Sh

Ck

'Ck+l

(42)

The scale-0 representation Co is the original input. Since thenumber of samples at each scale is reduced by a factor of 4
from that at the previous scale, the computational cost of repeating operations at multiple scales is greatly reduced. In
fact, this multiple-scale representation of the input is less than
4/3 as large as the original input. Note also that since mostprocessing occurs in the frequency domain, the shrink operation requires essentially no additional computation.
D. Input
The input is given as a movie C0 of normalized real floatingpoint values, with dimensions W X H X L. The movie dimensions can be mapped to world coordinates (degrees ofvisual angle, seconds) by a choice of scale factors. In the examples shown, the spatial scale was left undefined, but thetime scale was set at 80 frames/sec.

where fft indicates a fast Fourier transform. The result is acomplex movie Mo,1 of size W X H X L that defines the frequency response of the sensor up to W 2-1 cycles/width inboth u and v and 40 Hz in w. Figure 11b shows an exampleof the magnitude of the transfer function of a sensor.
3. Scaled Transfer Functions
The scale-h sensor transfer function is equal to the scale-h 1 transfer function minified by a factor of 2 in u and v dimensions. It can be obtained by subsampling the transformof the previous scale in u and v dimensions. We denote thissubsampling operation by the symbol Su (Fig. 16). In thisway, the sensor transfer functions for the various scales areeasily obtained from the transfer function at scale 0:
Su

Mk,l

4.

The input is Fourier transformed,
f ft _CM C&,(45)

yielding a complex movie of size W 2 -k X H 2 - X L. The
input and impulse response transforms are scalar multiplied,(46)

Mkj Ck = Rk,,

7
E. Summary of Linear Operations
The first stage of the model is a set of linear sensors that differ
in scale (spatial frequency), direction, and location. The response of a sensor of a particular direction, scale, and location
is determined by cross correlating the sensor weighting
function with the input. The responses of all sensors of aparticular direction and scale are computed by convolving thesensor impulse response and the input and by sampling theresult at the sensor locations. The convolution is done bytransforming the impulse response and the input to the fre-(44)

Mk+1,1lSensor Response

~~Su

.. D.

M.,

M ars,/,

Fig. 16. Illustration of the frequency-subsampling operator Su. The
array at each successive scale is obtained by sampling every otherelement in u and dimensions.

336

J. Opt. Soc. Am. A/Vol. 2, No. 2/February 1985

A. B. Watson and A. J. Ahumada, Jr.

and the result is inverse transformed, yielding the response
fft-l

Rkj

)- Rag.

(47)

The value at x, y, t in this real movie specifies the responseof a sensor of scale k and direction I at location x, y sampled
at time t. (An example is pictured in Fig. 20, below.)
5. Border Effects
The method of convolution used above implicitly assumes thatthe input is periodic in x and y. Hence the outputs at theborders are due in part to input from the opposite border.
These invalid results must be removed by stripping away aborder (N - 1)/2 elements wide from the output, where N is
the width of the nonzero extent (support) of the impulse response. Here N is approximately seven pixels independentof scale, so we remove a border three pixels wide from theoutput of each convolution (an alternative would be to ensurethat each input had a null border at least three pixels
wide).
F. Summary of Nonlinear Operations
The operations performed in the second stage of the model
are nonlinear. They combine responses of sensors within avelocity group, which up to this point have been kept separate.
(Recall that a velocity group is the set of sensors of the same
scale and location that differ in direction.) The principal
steps are(1) Measure the temporal-frequency spectrum of eachsensor response.
(2) Select the temporal frequency with the largest magnitude.
(3) Find which of each pair of opposite-direction sensors(within the same velocity group) has the larger magnitude,
save its frequency and magnitude.
(4) Within each velocity group, apply a fft to the vectorof temporal-frequency values, considered as a function of
direction.
(5) Use the amplitude and the phase of the first harmonicin the digital Fourier transform to calculate speed and direction, respectively.
(6) Save the largest magnitude within each velocity groupas a measure of the strength of the response. If this magnitude is less than a threshold, assign the velocity "undefined"
to the sensor.

the direction of image motion should be silent. But since inthe human visual system and in this implementation there areboth noise and approximation, the sensors in the null direction
may not be truly silent. Therefore we use a sort of inhibition
to cancel their responses. Within each direction group, we
compare the magnitudes of each pair of oppositely directed
sensors. The larger is preserved; the smaller is replaced withminus the larger. We do this rather than null the smaller only
to simplify the computations of Subsection 5.G.1. Finally,
we compare all thp magnitudes within the direction group andpreserve the largest. We will call this the strength of the direction-group response.
The result at this step is, for each location and scale, a movieof size Wk X Ha X (D + 1), where D is the number of directions (10). The array specifies, for each location, the peakfrequency as a function of direction (D values) and thestrength of the velocity-group response.
2. Estimating Speed and Direction
The frequency-versus-direction function at each location
should be a single cycle of a cosine with an amplitude equal
to the product of the sensor spatial frequency and the speedof image-component motion and with a phase equal to thedirection of image-component motion. The amplitude andthe phase are computed as the first-harmonic results of a ffton the frequency-versus-direction vector. The final output
for each scale is then a movie of size Wk X Ha X 3. The firstand second frames contain speed and direction estimates.
respectively, and the last frame contains the strength estimates. As noted in Subsection 5.E.5, a border of width 3 hasbeen assigned a strength of 0.
6.

RESULTS

We have applied our computational model to a small numberof test cases. The inputs were either digitally synthesized or:

G. Measuring Temporal Frequency
Since the spatial-frequency spectrum of the sensor is a
Gaussian, it seems likely that the temporal-frequency spectrum of the response to a broadband stimulus would beroughly Gaussian. The desired quantity would be the locationof this Gaussian. As a simple approximation to this, we havedetermined the frequency at which the largest magnitude
occurs. Thus at each scale, direction, and location, a fft isapplied to the vector of time samples of the sensor response;
The frequency with the largest response magnitude and themagnitude itself are saved. The result at this stage is, for eachscale k and for each direction, a movie of size Wk X Hk X 2.

10.I

1. Combining Responses within the Direction GroupIn theory, only half of the sensors should respond to a moving
stimulus: Those with directions more than 90 deg away from

A
-

X4

ien

Fig. 17. 3D Gaussian blob. The dimensions are W = 32, H = 32, L= 16. The speed is v¶ pixels/frame, and the direction is 3150. The
spatial spread is two pixels; the temporal spread is eight frames.

A. B. Watson and A. J. Ahumada, Jr.

g

S~~ff

Fig. 18.

Vol. 2, No. 2/February 1985/J. Opt. Soc. Am. A

l

_

X

00

t0VX

_

337

input (Fig. 18). Note that it consists of a Gaussian disk (broadin u and v, narrow in w) that has been sheared into a plane at
45°to both the u and the axes. The energy in the fourth|quadrant exists only for negative time frequencies (earlyframes in Fig. 18), while energy exists in the second quadrant
only for positive frequencies. This should be compared withthe amplitude response of the sensor pictured in Fig. lib.
It is also of interest to examine the response of the scalarsensors of one directed frequency (scale and direction) to theGaussian blob input. This is shown in Fig. 19. The response

Amplitude spectrum of the Gaussian blob. Th ajxes aras in Fig. lib.

A•~ ~,4,4
'~'

'7>'"

7>

t

~ ~~~ ~ ~ ~~ ~~~ ~
7/77>"

anrect1-36'.';«~

20. Rsoeoftevector sensors at scale 0 to the Gaussian7<,~4~,>@,7~~4St'Fig.

blob. Each arrow is an estimate of image velocity at the correA~~~~"j~~sponding spatial frequency and location. The contrast of the arrow~ ~~~~~nicates the strength of the response.

9 11

the spatial dimension smallto minimize execution time. The
length of each input was 16 frames (200 msec).

A .T hr e e -D im e n s io nsali Gs uon nt

i 00 l0t

i g . 1 7 . I t i s a 2 D sp ati al:00000

T Ga us si anwh osbe0coo nt sr asst0ov arni eis>as0iaj.Ga1u. si Ia fsu a ct2 o opf;0tti ame0i00000000000X000

1fg@;g000t0900E-

0C^0it00:-V >iQ !

t00^ >i;;f000f S;
0 .§00Z. f % 0 X0 B 00 0ii yst 000X
tt0000'00.0
0
0000W

0

0x

00

00;0 000

0 i000;i00000500
0.00 0 00 0 90;0 <0:t:0
00 as ittraverses the field from upper left to bottom right. This
? 090 : 40 7
5f 5
l00.
0
t
0
0
$
;
4
0
00
;0000
and
frequencies
is a useful input, because it contains all spatial:gXEV¢SEiE
0
all move in the same clear direction. It also can be adequately ggX t;0V;tt : ;ttXVtS::
using.-_
areweresolution
sampled at the low
Fig. 21. Vectr-sensor responses to the Gaussian blob at scale 1.
It is instructive to look at the amplitude spectrum of thisJ. Opt. Soc. Am. A/Vol. 2, No. 2/February 1985

338

N

N

A-

,,

-

4Z.

A,/:-

A-f-

A...-

A--.

/

'/

A-'

A/

,N

N
N

N,N

N

N

N

\

-

/

4_

4_

(.

/

/

Ad

A. B. Watson and A. J. Ahumada, Jr.

A-

4-

A-

4

of

A--__-

baFig. 22. Simulated responses to the sum of two gratings of different spatial frequencies that move in different directions. The frequencies
were 2 and 8 cycles/width, the directions were 900 and 1800, and the speeds were both 1 pixel/frame. a, Response at the scale of the lower frequency.
b, Response at the scale of the higher frequency.

a
½
A

½
A

4
S

Fig. 23. Simulation of apparent motion. The input was as shownin Fig. 17 with all but frames 6 and 8 blank. The output at a scale of
4 cycles/width is shown.

takes some time to develop and makes its first substantial
appearance in the upper-left-hand corner of frame 3. The
responses in the lower right-hand corner of the first four
frames are an artifact of the use of a circulant convolution
method and should be interpreted as frames 17-20 of the response, wrapped around into the first four frames. Note thatthe sensor passes only spatial frequencies near that of thesensor, with the result that, for this input, the response looks
much like the spatial weighting function of the sensor moved

along the path of the input. For sensors of other directions,the result would look the same, except that the amplitude of
the response and the orientation of the bars would be different.
Figure 20 shows the response of the vector sensors at scale
0. Since the input has a width of 32, this corresponds to a
spatial frequency of 8 cycles/width, the highest frequency(smallest scale) that we can compute on this input. Eacharrow represents the response of one vector sensor. The speed
is indicated by the length of the arrow and the direction by theangle of the arrow. In addition, the intensity of the arrow is
proportional to the strength of the sensor response. Arrows
have not been plotted whose intensity or length is less than
10% of the maximum within the picture.
The arrows point in the correct direction and have approximately the correct length. Furthermore, strong responses occur only along the actual path of motion. For this
input, at this scale, the model gives a good indication of thetrue and perceived velocity of image components.
Figure 21 shows the response at the next scale (4 cycles/
width). The results are similar. There are fewer arrows because the sampling density of the sensors is lower at this scale.
The responses are less confined to the path of motion, as wewould expect of sensors with lower spatial resolution.
B. Coherent Gratings
We have simulated the experiment of Adelson and Movshon
described in Subsection 1.D. We first superimposed two
gratings with the same spatial frequency (8 cycles/width)moving at 1 pixel/frame in directions of 90° and 1800. To thehuman observer, this appears as a coherent plaid that moves
to the upper left (135°). The model response at a frequencyof 8 cycles/width, where the sensors are matched to the spatial
frequency of the stimulus, correctly indicated motion at about
the right speed in about the right direction.
We then added together a grating of 8 cycles/width andA.
~~Vol.
B.Watson
2, No.
andA.2/February
J.Ahumada,Jr. 1985/J. Opt. Soc. Am. A

A. B. Watson and A. J. Ahumada, Jr.

339

direction of 900 and one of 2 cycles/width and direction 1800,
both with a speed of 1 pixel/frame. This appears to thehuman as a pair of separate gratings gliding over each other,each moving in its own direction. As shown in Fig. 22, themodel also indicates that each frequency should appear tomove in its own direction.
C. Apparent Motion
The input was a pair of Gaussian spatial blobs, as describedabove, that appeared only in frames 6 and 8 of the 16-frame
sequence. The distance between blob centers was two pixels

i t_

'I

_

.1

i S

1I4'I,

Fig. 24. A sequence of natural images in which 1two, objects (thehands) move in different directions. The width is 32 pixels.

I4~~~~~~~~~A

4

b
Fi.26
eto-esoOupua

fromtheascleof8

esoss
4cls

irstpoinyceswit.

tohescond

Fi.D. nVectr-snsogepne
inreiacho xandlyis
17 itho allbueramesig
1 3

~~~

4

1

Fig. 25. The scalar-sensor responses Rk,i to the input in Fig. 24.
Scale, 8 cycles/width, direction, 1080. The first four frames are
wrap-arounds from the end of the response.

ote

ad-aig,ututataThe

eunc.acae

fvetrsnsrotu

otehn-aigsqec.a

isthea smae sequence
inig.tzdfo
64Iw
andero Atvthe appropriate

340

J. Opt. Soc. Am. A/Vol. 2, No. 2/February 1985

It is an interesting case because it contains objects moving indifferent directions at different places. It is edifying to examine the responses of the scalar sensor for a particular scaleand direction. Figure 25 shows the scalar response Rk,! at ascale of 8 cycles/width in a direction of 1080. These sensors
respond only to the upward-moving right band, all other image
components having been filtered out.
The vector responses are shown in Fig. 26. The output at
both scales shows responses that are approximately correct
as to speed and direction (as compared with the nominal speed
and direction of the hand) and agree with our subjective impression of the movement of the hands.
7.

DISCUSSION

A. Relation to Other Models
Since our first descriptions of the scalar motion sensor,5 7 two
descriptions of similar mechanisms have appeared. 5 0'5' Bothmodels resemble ours in beginning with linear spatiotemporal
filters selective for spatial frequency, but both differ from ours
in important respects. In the van Santen-Sperling model,
the outputs of opposite-direction sensors are multiplied andthe result integrated over some interval. In the model of
Adelson and Bergen, the energy in each sensor output is integrated over some interval. Both of these procedures, which
we characterize as energy models, are fundamentally different
from the frequency-demodulation operation used in ourmodel. The energy models go to some effort to remove thetemporal modulation of the sensor response, whereas we
preserve it and note that it directly codes the image-velocity
components. It is interesting to note that energy models will
be quite susceptible to variations in the contrast of image
components at different orientations and directions. Ourmodel will, on the other hand, be immune to these effects,
since the temporal frequency of the scalar-sensor response islargely independent of contrast. This advantage is similar
to that enjoyed by FM radio broadcasts over AM.
The descriptions of the energy models do not yet go beyond
the level of a single scalar sensor. Both are defined in only one
spatial dimension and thus do not confront the problem of
estimating the full velocity vector from the ambiguous scalar-sensor responses. Neither model specifies the particularspatial or temporal impulse response of their sensors. Finally,both energy models are of a single sensor and hence do not
specify how the sensors are distributed over space or frequency. These differences prevent further comparisons with
our model, but it seems likely that experimental tests might
distinguish between our model and more completely specified
versions of the energy models.
B. Remaining Questions
We have constructed a system for processing a dynamic visual
input (a movie), which assigns velocities to components of the
input. The output may be regarded as a sampled, multiplescale velocity field. The velocity assignments made by thesystem resemble those made by human observers, though nostrict tests of this correspondence have yet been made.
A number of issues have been left unresolved, and several
known features of human vision have not been modeled. For
example, though our temporal impulse response is matchedto human sensitivity, no similar match has been made in theA. B. Watson and A. J. Ahumada, Jr.

spatial domain. The spatial contrast-sensitivity function willaffect the precise correspondence between model outputs andhuman judgments. Likewise, the space-variant resolutionof human vision may have important effects on motion perception, but it is not included in this version of our model.
Another intriguing and unresolved issue is the time scaleof velocity assignments. There are at least four questions:(1) How much time is used to compute a velocity estimate?
(2) How often is a new estimate computed?
(3) Are the answers to the previous two questions the sameat each scale?
(4) To what moment in time is the velocity assigned?
The first three questions are straightforward and might be
resolved by experiment, but the fourth requires further explanation.
At issue is how information from different sources is related
in the time dimension. Just as estimates of velocity are assigned to particular locations and scales, so are they assigned
to particular times. Just as we might suspect that information
at different scales is kept in registration by a topographic
mapping, so information from different mental processing
channels must be kept in chronographicregistration. For
instance, to catch a ball in flight the human must keep inregistration a number of visual representations (e.g., velocity,
shape, color) as well as a motor representation of the positionof the hand.
This example also illustrates the special problems in perception that are due to the causality of processing in the time
domain. Visual processing takes time and must therefore lag
behind the stimulus events in the world. But, to catch theball, one must abolish this lag through prediction and anticipation. Therefore we must suppose that, based on a mental
clock, velocities are assigned to the past and used to predictthe present and future.
As noted in the introduction, a model of human motion
sensing must pass two tests. The first is whether it successfully mimics psychophysial data on sensitivities, discriminations, illusions, and appearances. It has passed some preliminary tests of this sort, as described in Section 6. The
second test is how useful and efficient the representation isfor further mental computations. This is likely to be answered only by trying to model higher visual functions, suchas image segmentation, object recognition, and judgments ofobject and self-motion in four-dimensional space-time.

REFERENCES
1. J. J. Gibson, The Perception of the Visual World (Houghton

Mifflin, Boston, Mass., 1950).
2. S. Ullman, The Interpretationof Visual Motion (MIT U. Press,

Cambridge, Mass., 1979).
3. H. C.Longuet-Higgins and K. Prazdny, "The interpretation ofmoving retinal images," Proc. R. Soc. London Ser. B 208, 385-387(1980).
4. E. C. Hildreth, The Measurement of Visual Motion (MIT U.

Press, Cambridge, Mass., 1983).
5. A.B. Watson and A.J. Ahumada, Jr., "A look at motion in thefrequency domain," NASA Tech. Mem. 84352 (1983).
6. E. H. Adelson and J. A.Movshon, "Phenomenal coherence ofmoving visual patterns," Nature 300, 523-525 (1982).

A. B. Watson and A. J. Ahumada, Jr.
7. A.B. Watson and A. J. Ahumada, Jr., "A linear motion sensor,"
Perception 12, A17 (1983).
8. F. W. Campbell and J. G. Robson, "Application of Fourier analysis
to the visibility of gratings," J. Phys. London 197, 551-566(1968).
9. N. Graham and J. Nachmias, "Detection of grating patterns
containing two spatial frequencies: a comparison of singlechannel and multiple-channel models," Vision Res. 11,251-259(1971).
10. A. B. Watson, "Summation of grating patches indicates many
types of detector at one retinal location," Vision Res. 22, 17-25(1982).
11. N. Graham, "Spatial-frequency channels in the human visualsystem: effects of luminance and pattern drift rate," Vision Res.
12, 53-68 (1972).
12. A.B. Watson, P. G.Thompson, B. J. Murphy, and J. Nachmias,"Summation and discrimination of gratings moving in opposite
directions," Vision Res. 20, 341-347 (1980).
13. P. Thompson, "The coding of velocity of movement in the humanvisual system," Vision Res. 24, 41-45 (1984).
14. S. P. McKee, "A local mechanism for differential velocity detection," Vision Res. 21, 491-500 (1981).
15. A. Pantle, "Motion aftereffect magnitude as a measure of thespatiotemporal response properties of direction-sensitive analyzers," Vision Res. 14, 1229-1236 (1974).
16. E. Levinson and R. Sekuler, "Adaptation alters perceived direction of motion," Vision Res. 16, 779-781 (1976).
17. D. J. Tolhurst, "Separate channels for the analysis of the shapeand the movement of a moving visual stimulus," J. Physiol.
London 231, 385-402 (1973).
18. E. Levinson and R. Sekuler, "The independence of channels inhuman vision selective for direction of movement," J. Physiol.
London 250, 347-366 (1975).
19. C. F. Stromeyer III, J. C. Madsen, S. Klein, and Y. Y. Zeevi,"Movement-selective mechanisms in human vision sensitive tohigh spatial frequencies," J. Opt. Soc. Am. 68, 1002-1005(1978).
20. R. J. W. Mansfield and J. Nachmias, "Perceived direction of
motion under retinal image stabilization," Vision Res. 21,
1423-1425 (1981).
21. J. G.Robson, "Spatial and temporal contrast-sensitivity functionsof the visual system," J. Opt. Soc. Am. 56, 1141-1142 (1966).
22. J. J. Koenderink and A. J. van Doorn, "Spatiotemporal contrast
detection threshold surface is bimodal," Opt. Lett. 4, 32-34(1979).
23. D. H. Kelly, "Adaptation effects on spatio-temporal sine-wave
thresholds, " Vision Res. 12, 89-101 (1972).
24. D. H. Kelly, "Motion and vision. II. Stabilized spatio-temporaltheshold surface," J. Opt. Soc. Am. 69, 1340-1349 (1979).
25. D. C. Burr and J. Ross, "Contrast sensitivity at high velocities,"
Vision Res. 22,479-484 (1982).
26. P. A. Kolers, Aspects of Apparent Motion (Pergamon, New York,1972).
27. G. Sperling, "Movement perception in computer-driven visualdisplays," Behav. Res. Methods Instrum. 8, 144-151 (1976).
28. A.B. Watson and A. J. Ahumada, Jr., "Sampling, filtering, andapparent motion," Perception 1, A15 (1982).
29. A. B. Watson, A. J. Ahumada, Jr., and J. Farrell, "The windowof visibility: a psychophysical theory of fidelity in time-sampledvisual motion displays," NASA Tech. Pap. 2211, (1983).
30. M. Fable and T. Poggio, "Visual hyperacuity: spatiotemporalinterpolation in human vision," Proc. R. Soc. London Ser. B 213,
451-477 (1981).

Vol. 2, No. 2/February 1985/J. Opt. Soc. Am. A

341

31. A. B. Watson and J. G. Robson, "Discrimination at threshold:labelled detectors in human vision," Vision Res. 21, 1115-1122(1981).
32. E. Levinson and R. Sekuler, "Inhibition and disinhibition of direction-specific mechanisms in human vision," Nature 254,
692-694 (1975).
33. C. F. Stromeyer III, R. E. Kronauer, J. C. Madsen, and S. A. Klein,"Opponent-movement mechanisms in human vision," J. Opt. Soc.
Am. A 1, 876-884 (1984).
34. P. Thompson, "Discrimination of moving gratings at and above
detection threshold," Vision Res. 23, 1533-1538 (1983).
35. F. W. Campbell, J. Nachmias, and J. Jukes, "Spatial-frequency
discrimination in human vision," J. Opt. Soc. Am. 60, 555-559(1970).
36. F. H. C. Crick, D. C. Marr, and T. Poggio, "An information processing approach to understanding' the visual cortex," in The
Organizationof the CerebralCortex, S. G. Dennis, ed. (MIT U.
Press, Cambridge, Mass., 1981).
37. J. G. Moik, "Digital processing of remotely sensed images," NASADoc. SP-431 (1980).
38. The orientation of a 2D frequency component is the angle of anormal to the wavefront.
39. D. Marr and S. Ullman, "Directional selectivity and its use in earlyvisual processing," Proc. R. Soc. London Ser. B 211, 151-180(1981).
40. F. L. van Nes, J. J. Koenderink, H. Nas, and M. A. Bouman,"Spatiotemporal modulation transfer in the human eye," J. Opt.
Soc. Am. 57, 1082-1088 (1967).
41. A. B. Watson, "Temporal Sensitivity," in Handbook of Perceptionand Human Performance,J. Thomas, ed. (Wiley, New York, tobe published).
42. M. G. F. Fourtes and A. L. Hodgkin, "Changes in the time scaleand sensitivity in the omatidia of limulus," J. Physiol. London
172, 239-263 (1964).
43. A. B. Watson, "Derivation of the impulse response: commentson the method of Roufs and Blommaert," Vision Res. 22,
1335-1337 (1982).
44. S. Marcelja, "Mathematical description of the responses of simple
cortical cells," J. Opt. Soc. Am. 70, 1297-1300 (1980).
45. B. Sakitt and H. B. Barlow, "A model for the economical encodingof the visual image in cerebral cortex," Biol. Cybern. 43, 97-108(1982).
46. A.B. Watson, "Detection and recognition of simple spatial forms,"
in Physicaland Biological Processingof Images, A. C. Slade, ed.
(Springer-Verlag, Berlin, 1983).
47. S. Tanimoto and T. Pavlidis, "A hierarchical data structure for
picture processing," Comput. Graphics Image Process. 4, 104-119(1975).
48. P. J. Burt and E. H. Adelson, "The Laplacian pyramid as acompact image code," IEEE Trans. Commun. COM-31, 532-540(1983).
49. J. L. Crowley and R. M. Stern, "Fast computation of the difference of low-pass transform," IEEE Trans. Pattern Anal. Mach.
Intelligence PAMI-6, 212-222 (1984).
50. G. Sperling and J. P. H. van Santen, "Temporal covariance modelof human motion perception," J. Opt. Soc. Am. A 1, 451-473(1984).
51. E. H. Adelson and J. R. Bergen, "Motion channels based on
spatiotemporal energy," Invest. Ophthalmol. Vis. Sci. Suppl. 25,
14 (A) (1984).
(see overleaf)

342

J. Opt. Soc. Am. A/Vol. 2, No. 2/February 1985

A. B. Watson and A. J. Ahumada, Jr.

Albert J. Ahumada, Jr.

Andrew B. Watson
Andrew B. Watson did undergraduate
work at Columbia University and in 1977
received the Ph.D. degree in psychology
from the University of Pennsylvania. He
spent three years in England at the Craik
Laboratory of Cambridge University,
followed by two years in the Department
of Aeronautics and Astronautics at
Stanford University. He has been at
NASA Ames Research Center since 1982,
where he heads the Perception and Cognition Group. Current research interests
are computational models of visual perception and their application to image coding, analysis, and display.

I

Albert J. Ahumada, Jr., received the B.S.
degree in mathematics from Stanford
University in 1961 and the Ph.D. degreein psychology from the University ofCalifornia at Los Angeles in 1967. He hastaught at the University of California,
Irvine, in the School of Social Sciencesand done research in the Department of
Aeronautics and Astronautics at Stanford
University. He is now a research psychologist at NASA Ames Research Center, where his primary research interest is
the development of mathematical modelsof visual processes.