Activation of the Human Superior Temporal Gyrus during
Observation of Goal Attribution by Intentional Objects
Johannes Schultz1, Hiroshi Imamizu2,
Mitsuo Kawato2, and Chris D. Frith1

Abstract& Previous functional imaging experiments in humans showed
activation increases in the posterior superior temporal gyrus
and sulcus during observation of geometrical shapes whose
movements appear intentional or goal-directed. We modeled achase scenario between two objects, in which the chasing object
used different strategies to reach the target object: The chaser
either followed the target’s path or appeared to predict its end

INTRODUCTION
A well-studied phenomenon of visual perception inhumans is the ability to recognize the movements ofmoving living beings, even if these movements arepresented in impoverished displays such as the moving
dots of Johansson’s point-light displays (Johansson,1973). But a single moving dot can also appear animate.
Depending on the way it moves, it can even appear
intentional, wanting something, or courting anotherobject (Klin, 2000; Blythe, Todd, & Miller, 1999; Bassili,
1976; Heider & Simmel, 1944). Some characteristics
that are necessary for a moving object to appear animate have begun to emerge from behavioral studies inchildren and adults. When the object’s movement does
not appear to be explainable by simple physical rules
such as gravity, conservation of the movement quantity,
or magnetic force, the object appears self-propelled,which makes it more likely to appear animate (Scholl& Tremoulet, 2000). The impression of animacy increases if the object appears to move contingently withthe movements of a human (Johnson, 2003). A particularly important factor appears to be goal-directed motion, defined as a ‘‘type of autonomous movement inwhich the agent contingently directs its movement
toward (or away from) another object, state or location’’(Csibra, 2003; Opfer, 2002).
Psychophysical studies suggest that detection of biological motion is probably effected by very f lexible

1
2

Wellcome Department of Imaging Neuroscience, London, UK,
ATR Computational Neuroscience Laboratories, Kyoto, Japan

D 2004 Massachusetts Institute of Technology

position. Activation in the superior temporal gyrus of humanobservers was greater when the chaser adopted a predict rather
than a follow strategy. Attending to the chaser’s strategy
induced slightly greater activation in the left superior temporalgyrus than attending to the outcome of the chase. These data
implicate the superior temporal gyrus in the identification of
objects displaying complex goal-directed motion. &mechanisms under the influence of learning, different
from those involved in the detection of other forms ofcomplex motion (Giese & Poggio, 2003; Neri, Morrone,& Burr, 1998). Neuroimaging and neurophysiological
data suggest that the cortex surrounding the posterior
part of the superior temporal sulcus and gyrus can be
activated by biological motion (Puce & Perrett, 2003;
Bonda, Petrides, Ostry, & Evans, 1996; Perrett et al.,1985) such as point-light displays of moving humans andanimals. Neuroimaging experiments show that this region is also activated by simple moving objects whose
interactions appear causal, goal-directed, or intentional(Blakemore, Boyer, et al., 2003; Schultz et al., 2003;
Blakemore, Fonlupt, et al., 2001; Castelli, Happe, Frith,& Frith, 2000), and also during more cognitive processes
such as attribution of mental states to others (Saxe &Kanwisher, 2003; Frith & Frith, 1999). Observation andimitation of goal-directed human actions also elicit
activation increases in this region of the brain (Grezes,
Armony, Rowe, & Passingham, 2003; Iacoboni et al.,
2001; Winstein, Grafton, & Pohl, 1997).
Recognizing goal-directed motion is a necessary step
towards attribution of intentions to other agents andappears to be identified by the brain whether the agent
producing it is a human being or an animate-looking
moving object. Previous data from our group show thatobservation of abstract geometrical shapes moving in agoal-directed way induces activation increases in the
superior temporal sulcus, activation increases further
when the agent appears to act intentionally (Castelli
et al., 2000). More recent, unpublished data show that
activation in the superior temporal sulcus and gyrus

Journal of Cognitive Neuroscience 16:10, pp. 1695–1705

increases in relation to the degree of correlated motion
between the two abstract moving objects. We hypothesized that observation of a moving geometrical shapechasing another moving shape might induce greater
activation increases in the superior temporal gyrus andsulcus when the chasing object appears to understandthe goals of the target object and predict its movements
rather than simply following the target. Consequently,
our chasing object used one of two strategies to catchthe target object: It either followed the target on its path(‘‘follow’’ strategy), or it went directly to the end point of
the other object’s trajectory (‘‘predict’’ strategy), as if it
had predicted where the target would go. When thechasing object used the predict strategy rather than thefollow strategy, it gave the impression that it was able toinfer the goal of the other object’s movement. In theexperimental design, this ‘‘strategy’’ factor was crossedwith an ‘‘outcome’’ factor: The chase was only successful
in 50% of cases, allowing the performance of a second
task using the same animations.
Previous studies indicate that activation in the superior temporal gyrus area increases when subjects attend

to more ‘‘socially relevant’’ dimensions of a visualdisplay, such as emotion (Narumoto, Okada, Sadato,
Fukui, & Yonekura, 2001), trustworthiness (Winston,
Strange, O’Doherty, & Dolan, 2002), and contingency
between the movements of two objects (Blakemore,Boyer, et al., 2003). We therefore asked participants in
our experiment to perform two different tasks using
the same stimuli. In one half of the experiment,
participants were asked to decide which strategy the‘‘chasing’’ object used to catch the ‘‘target’’ object (the‘‘strategy’’ task); in the other half of the experiment,
subjects had to judge whether the chaser was successful in reaching the target objects (the ‘‘outcome’’ task).
Examples of conditions in the experiment are displayedin Figure 1.

RESULTS
Behavior
Analysis of participants’ ratings from the strategy task
and the outcome task performed in the scanner showedthat participants performed well in both the strategy

Figure 1. Design and stimuli.
The open light gray arrow
indicates time at which thedark gray object started to
move. Four experimental
conditions come from the
combination of two factors
in the experiment: predict
strategy versus follow strategy
and catch outcome versus nocatch outcome. Below on theleft is the specific control
condition for the follow catch
condition; such specificcontrol conditions existed
for all four experimental
conditions, resulting in eight
conditions in total. Subjects
categorized either the strategy
or the outcome of the blue
object’s movement in separate
tasks on the same animations,
therefore the Total numberof conditions  Task
combinations in theexperiment is 16. The objects
moved to any of the four
corners of the screen withequal frequency. The object
depicted in light gray waspresented in red during theexperiment, the object
depicted in dark gray was bluein the experiment.

1696

Journal of Cognitive Neuroscience

Volume 16, Number 10

and the outcome task. Subjects had to perform thetasks only when a chase between the two objects took
place, and they correctly identified the strategy of
the chasing object in 94% of ‘‘predict’’ trials and 91% of‘‘follow’’ trials (±3% SEM over subjects). Positive outcome (i.e., a successful chase) was identified correctly
in 89% of trials (±4) and negative outcome (i.e., an
unsuccessful chase) was identified correctly in 79% of
trials (±5.6). There were no significant differences
between ratings of predict and follow trials (pairedt test, p > .05), but positive outcome trials were more
often identified correctly than negative outcome trials(paired t test, p < .05). This difference probably stems
from subjects responding too quickly and miscategorizing some negative outcome trials as positive outcome
trials. This difference does not influence the interpretation of the brain activation analysis because trials withdifferent outcomes were not compared with each otherand only served as an implicit task. When asked todescribe the stimuli, subjects typically reported that thechasing object ‘‘appeared to know where the target
object was going’’ (predict trials), appeared to follow it(follow trials) or to move in opposite directions (control trials). They reported that they tried to find out
how the chasers tried to reach their goal in the strategy
task and that they tried to judge if the chaser had
reached the target in the outcome task. All subjects
described both tasks as being quite easy.

fMRI Data
Voxelwise Analysis in the Superior Temporal
Sulcus and Gyrus
We used the factorial design of this experiment totest for activation differences due to the strategy usedby one object to chase another object and to test
how these activation differences were affected by theattentional task the subject performed. To discount object motion effects, we assessed interaction-sensitive
activations as the difference between the activation increases due to a predict versus a follow strategy in chasetrials and the same activation increases in the equivalent
movement control trials. Based on previous studies of
biological motion and interacting geometrical shapes,
we were particularly interested in the activation of thecortex in the posterior part of the superior temporalsulcus and gyrus of both hemispheres.
Effects of Strategy
Activation increases were found in the superior temporal gyrus and the superior temporal sulcus of bothhemispheres when subjects observed a chasing object
with a predict strategy compared to observation of achasing object with a follow strategy (Figure 2). This wastested formally with a two-way interaction between

predict versus follow strategies and chase versus control
trials to discount differences in object movement. The
opposite contrast yielded no significant activation increases in the superior temporal sulcus or gyrus. Neither
contrast yielded activation in the medial prefrontal cortex or the temporal poles, even at a relatively low significance threshold ( p < .01, uncorrected, T > 2.65).
Effects of Task on Strategy. When tested as interactions between strategy and task, no cluster showedsignificantly different activation depending on the task
the subjects performed. This was tested by the following
three-way interactions: interaction between task (strategy vs. outcome), strategy (predict vs. follow), and chaseversus control trial, and interaction between task (outcome vs. strategy), strategy (predict vs. follow), andchase versus control trial. However, a cluster of voxels
in the left posterior superior temporal sulcus was foundwhose activation was greater during observation of apredict versus a follow strategy when subjects performed the strategy categorization task, but was not
significantly activated when subjects performed the outcome task. When we masked the results of the comparison in the strategy task with those of the outcome task,
the cluster in the left posterior superior temporal sulcus
identified in the strategy task remained significantly
activated (Figure 3A; exclusive mask, threshold p =.05; activation remains significant with all thresholds
below p = .15). This suggests that voxels in this cluster
respond only when subjects performed the strategy task.
Masking the results of the comparison in the outcome
task with those of the strategy task yielded a cluster ofvoxels in the right middle temporal gyrus, located caudal
and ventral to the cluster of activation in the posterior
superior temporal sulcus mentioned above. This cluster
was not located within the superior temporal sulcus andgyrus search regions (Figure 3B; exclusive mask, threshold p = .05; similar results with all thresholds below p =.15). Again, there were no significant activations in either
the medial prefrontal cortex or temporal poles, even at
the lower threshold of p < .01 uncorrected.

Activation Time Courses
Activation time courses in the voxels of the superior
temporal sulcus and gyrus with strongest activation
increase during observation of a predict versus a follow
strategy confirmed the results of the voxelwise analysis.
In the superior temporal sulcus in the left hemisphere,
activation during the initial phase of the trial was higher
in predict trials than in follow trials, but only whensubjects performed the strategy task [two-way ANOVA:
interaction between strategy and time: F(1,13) = 7.8,
p =.015]. In the right hemisphere, activation during thewhole trial was higher in predict trials than in follow
trials when subjects performed the strategy task [mainSchultz et al.

1697

Figure 2. Clusters in the
superior temporal sulcus andgyrus whose activation was
significantly greater duringobservation of a predict
strategy than a follow
strategy. Tested formally witha two-way interaction
between predict versus follow
strategies and chase versus
control trials. For details, see
Table 1. For display, theimage is thresholded at
p < .001, uncorrected for
multiple comparisons.
Clusters surviving a thresholdof p < .05, corrected for
multiple comparisons acrossanatomical search regions,are circled in blue, listed inTable 1, and discussed in
the text. Parameter estimatesfrom the voxels of maximum
intensity from these clusters
are displayed (Left STG, Right
STG, and MTG). P and F
represent predict and follow
conditions, and colors blue
and red represent strategy
and outcome tasks,
respectively. Parameter
estimates shown are fromchase conditions from which
the parameter estimates ofthe matched control
conditions have been
subtracted. Error bars
represent SEM over subjects.
The brain slices used for
display are the average image
from the subjects’ normalized
structural brain scans(see Methods). Coordinates
are in MNI reference space.
The color bar refers toactivation intensity expressedin t values.

effect of strategy: F(1,13) = 4.9, p = .045], and was only
initially higher when subjects performed the outcome
task [interaction between strategy and time: F(1,13) =5.6, p = .035].

DISCUSSION
This study shows that activation in the superior temporal gyrus and in the cortex surrounding the superior
temporal sulcus increases when a simple object appears
to chase another object by understanding the target’s
goal and predicting its movement instead of simply
following it. Directing attention to the object’s strategy

1698

Journal of Cognitive Neurosciencerather than to the outcome of its chase increased activation differences in the left superior temporal sulcus.
Attribution of Intentions and Goal-Directed
Movements and the Superior Temporal
Sulcus and Gyrus
A number of studies have shown activation increases in
the posterior part of the superior temporal sulcus andgyrus (together with the temporal poles and medial
prefrontal cortex) during attribution of mental states
to other agents, also called mentalizing or theory ofmind (Frith & Frith, 1999, 2003). The exact mechanism
on which this ability is based is not fully determined yet,

Volume 16, Number 10

but intuitively, a necessary first step towards the attribution of mental states to an agent could be theattribution of goals to this agent. When moving objects
appear to follow goals, they are more likely to beconsidered animate by human observers (Csibra, 2003;
Opfer, 2002). Previous studies show that activation in
the superior temporal sulcus and gyrus increases duringobservation of geometrical shapes if their movements
appear goal-directed or intentional (Castelli et al., 2000).
Recent unpublished data show that activation in thispart of the cortex increases parametrically with theamount of interactive motion of two objects. These datasuggest that this part of the cortex is sensitive to
movement characteristics associated with living entities,
especially goal-directed movements. In our present
study, participants observed an object understanding
the goals of another object and predicting its movements or simply following it in order to catch it. As
predicting the movements of a target object is a characteristic associated with a potential agent, the greater
activation in the superior temporal sulcus and gyrus
when the chasing object predicted the movements ofthe target object compared to simply following it suggests that this brain region plays a role in the identification of intentional agents.
Attention Effects
The cluster identified in the left superior temporal gyrus
showed greater activation increases in predict trials compared to follow trials when subjects tried to identify
the strategy of the chaser than when they assessed theoutcome of the chase (Figure 3A). As discussed in the
Introduction, activation increases in the superior temporal gyrus area have been reported in the right hemispherewhen attending to the emotion or the trustworthiness of
faces, and in the left hemisphere when attending tocontingency between the movements of abstract objects(Blakemore, Boyer, et al., 2003). Blakemore and colleagues suggested that the attention effects they observed could be due to top-down effects related to thesearch for agents, rather than bottom-up detection on thebasis of visual cues. In our case, attending to the strategies
used by the objects to reach their goal cued subjects into
paying attention to cognitive aspects of the objects’
movements. This task is much closer to searching foragents than simply comparing the positions of the chaser
to the position of the target at the end of the trial in theoutcome task. We suggest that the active process ofsearching either for agents or for characteristics of their
movements can increase or prolong neural responses
in areas already sensitive to intentional agents.
Time Course: Early Activations versus ConstantFigure 3. Clusters in the superior temporal sulcus and gyrus whose
activation was significantly greater during observation of a predict
strategy than a follow strategy and was affected by the subjects’ task.
(A) Activations in the strategy task masked with activation in theoutcome task (exclusive mask, threshold of mask: p = .05,
uncorrected). (B) Activations in the outcome task masked withactivation in the strategy task (exclusive mask, threshold of mask:
p = .05, uncorrected). The cluster circled in panel A survived athreshold of p < .05, corrected for multiple comparisons acrossanatomical search regions, was listed in Table 1 and discussed in the
text. The cluster circled in panel B was activated to a similar degree but
was not located within the search regions. The brain slices, coordinate
system, and color bar index are similar to Figure 2.

In the left hemisphere, activation during the initial phaseof the trial was higher in predict trials than in follow trials
when subjects performed the strategy task; in the righthemisphere, a similar pattern was observed during theoutcome task (Figure 4). Activation in the right hemisphere was higher in predict trials than in follow trials
during the whole trial when subjects performed thestrategy task. In the two cases where the activation was
only higher in predict than in follow trials at the beginning of the trial, plots of the data show that the activation difference was present at 6.75 sec after trial onset(Figure 4, top left and bottom right), which is compatible
with a neural event occurring at the onset of the trial. At
the next sampled time point (11.25 sec), the activationdifference in the left hemisphere was less important andit disappeared in the right hemisphere. We suggest that
this reflects an initial response of the cortex to ouranimations, which is increased and/or prolonged whensubjects pay attention to a socially relevant dimension of
the objects’ movement. The time course analyses suggestthat this response is stronger in the right hemisphere,
but as the statistical parametric map (SPM) analyses do
not confirm this tendency, we cannot argue strongly infavor of hemispheric specialization in this task.

Schultz et al.

1699

Figure 4. Activation timecourses in the superior
temporal gyrus in both
strategy and outcome tasks.
Solid lines refer to trials inwhich the chasing object
used a predict strategy,
dotted lines refers to trials inwhich the chasing object
used a follow strategy. The
thick black line on thehorizontal axis represents
the duration of theanimation. Activation isdisplayed in arbitrary units,
0 refers to average activation
during the whole course of
the experiment. Error bars
represent standard errors ofthe mean across subjects. In
the left hemisphere,
activation during the initial
phase of the trial was higher
in predict trials than infollow trials, but only whensubjects performed thestrategy task [two-way
ANOVA: interaction between strategy and time: F(1,13) = 7.8, p = .015]. In the right hemisphere, activation during the whole trial was higher
in predict trials than in follow trials when subjects performed the strategy task [main effect of strategy: F(1,13) = 4.9, p = .045], and was only initiallyhigher when subjects performed the outcome task [interaction between strategy and time: F(1,13) = 5.6, p = .035]. Data are from the peaksof activation (coordinates: left hemisphere: 54 34 4, right hemisphere: 48 44 12, MNI reference frame) of the significant clusters in the superior
temporal gyrus, identified with the predict versus follow * chase versus control contrast.

No Activation Increases in the Medial
Prefrontal Cortex
It is interesting to note that we did not find significant
activation increases in the medial prefrontal cortex or the
temporal poles in all the contrasts we tested, areas
which, together with the superior temporal sulcus, arethought to be involved in the attribution of mental states(Frith & Frith, 2003). Previous PET and fMRI studies
using complex animations leading to attribution of mental states have shown activation in these areas (Schultz
et al., 2003; Castelli et al., 2000), but studies using simple animations have not (Blakemore, Boyer, et al., 2003;
Blakemore, Fonlupt, et al., 2001). The latter studies usedanimations with objects interacting causally, appearing
animate, and/or interacting contingently, which are notthought to lead to attribution of mental states. In ourpresent study, observers watching an object attributing
a goal to another object did not show a significantly
greater activation in the medial prefrontal cortex thanwhen watching an object following another object, regardless of the task the subjects were performing. We
suggest that subjects solved this simple task with only asmall mentalizing effort or none at all, which was not
sufficient to induce activation in the medial prefrontal
cortex. Although activation increases in the superior
temporal sulcus could be related to mentalizing, we
would rather suggest that these are due to detection of

1700

Journal of Cognitive Neuroscience

intentional entities, which could explain why activationin the other areas involved in mentalizing was notincreased. This view is consistent with informal behavioral reports from our subjects and the results andinterpretation of a previous study (Blakemore, Boyer,
et al., 2003).

Differences in Movement Paths
A possible alternative explanation of activation differences caused by the observation of objects with different
movement paths could be the difference in complexity
between the paths. In the present experiment, this wascontrolled by matched control conditions in which thepaths of both chasing and target objects were identical tothe paths in the chasing trials except for the fact that thechasing object did not move towards the target object
but in the opposite direction. This control condition only
affected the contingency between the objects’ paths andnot the movement paths themselves. Objects moving in
opposite directions could be seen to be avoiding insteadof approaching each other, and therefore represent
another form of social interaction, which could also
induce activation in the superior temporal sulcus. Although this is a very valid possibility, in addition toobserving activation increases in the superior temporalsulcus and gyrus in the interaction between chase versus

Volume 16, Number 10

control and predict versus follow strategy, we also foundactivation in these clusters of activated voxels to behigher in predict than in follow trials when we subtracted
activation in the corresponding control conditions fromthe chase trial activation (see parameter estimate plots in
Figure 2). These data support our interpretation suggesting that observation of goal attribution by the chaser
induces activation increases in the superior temporalsulcus and gyrus, and that attention to the object’s
strategy further increases this effect.
Specialization along the Superior
Temporal Sulcus?
The cortex in the superior temporal sulcus and gyrus
responds to (i) the attribution of mental states to others,(ii) the identification of biological motion, and (iii) the
response to animate-looking, moving geometricalshapes. Whether different parts of the cortex in the sulcus
and gyrus are specialized for each of these functions is not
yet known. Against the idea of specialization is the factthat activation increases during observation of biologicalmotion have been found all along the superior temporalsulcus (Puce & Perrett, 2003) and do not appear organized in a particular way. Instead, a number of studies of
all three types of processes have all yielded activationin the same area of the superior temporal gyrus andsulcus (Figure 5). The activation increases found in thepresent study are also located in this area. This suggeststhat this region of the cortex could be involved in acommon aspect of these three tasks, for example, the response to goal-directed movements, whether expressedby moving abstract shapes or human beings.

METHODS
Subjects
Fourteen right-handed participants (8 men and 6 women, aged 19–40) participated in the study. All subjects
gave full written informed consent and the study wasapproved by the local ethics committee.
Stimuli, Animations, Design, and Tasks
Participants in the scanner watched short animations(4.2 sec per animation) in which two moving disks
appeared to be either interacting or to move independently from each other (Figure 1). The moving objects
were two disks or balls, with a width on the screen in thescanner of 28 of visual angle (Figure 1, top left). One wascolored bright red, the other bright blue, and the background was black. The movement trajectories of thetwo disks were determined prior to the experiment by
an equation of motion implemented in Matlab (TheMathWorks, Natick, MA). The visual display was controlled by in-house presentation software (Cogent 2000,
www.vislab.ucl.ac.uk/Cogent2000/index.html) imple-

Figure 5. Comparison of
results of the current study
with 16 previous studies ofmentalizing, biological motion,and moving geometricalshapes. In red and cyan are
results of the current study inthe left and right hemispheres,
respectively. In black areresults of the following
previous studies: Schultz et al.,
2003; Campbell et al., 2001;
Vogeley et al., 2001; Brunet,Sarfati, Hardy-Bayle, & Decety,2000; Castelli, Happe, Frith, &Frith, 2000; Downar, Crawley,
Mikulis, & Davis, 2000;
Gallagher, Happe, Brunswick,Fletcher, Frith, & Frith, 2000;
Grossman et al., 2000;
Hoffman & Haxby, 2000;
Grezes & Costes, 1998; Puce,
Allison, Bentin, Gore, &McCarthy, 1998; Wicker,
Michel, Henaff, & Decety,1998; Bonda, Petrides, Ostry, &Evans, 1996; Fletcher et al.,
1995; Goel, Grafman, & Hallett,1995. Displayed on Maximum
Intensity Projection glass brain
with SPM2.

Schultz et al.

1701mented in Matlab. Animations were projected onto an
opaque surface in the scanner by an LCD projector,
subjects viewed them through a mirror mounted on
the headcoil.
Design, Conditions, And Tasks
In this experiment, we wanted to test whether moving
objects elicit stronger activation in the posterior superior temporal sulcus and gyrus when they appear to
attribute goals to a target object than they appear tochase their target without attributing goals to it. We useda chasing situation where one object tried to catch another object by using one of two strategies. The chasing
object appeared to attribute a goal to the target object
by heading towards the end point of the target’s position, or simply followed it (no goal attribution). We usedmatched control conditions without chasing for eachexperimental condition to control for differences inobject motion across conditions. We also wanted to test
whether activation changes happen only when subjects
pay attention to the strategy used by the objects or also
when they perform another, incidental task. In thestrategy task, subjects categorized the chaser’s strategy
into ‘‘following’’ or ‘‘using prediction of the trajectory
end point.’’ In the outcome task (the incidental task),
they had to decide whether the chaser caught the target at the end of the trial or not. During the control
trials, subjects were asked to press one of the two response buttons at random. We therefore used a factorial
design with the following four factors: (1) two strategies
for the chasing object: predicting or following; (2) two
outcome levels: target caught or missed; (3) two task
levels: judging strategy or judging outcome; and (4) two
condition levels: chase and control (no chase). Combinations of all these factors resulted in 16 different trial
types, each repeated 24 times during the experiment,
for a total scanning time of 19.5 min. Subjects responded by pressing one of two buttons on a keypad
with the corresponding finger of the right hand. Buttonpresses and brain activation were recorded simultaneously during the experiment. On these two types ofdependent variable we assessed the effects of strategy
and the interaction between task and strategy.
Animation Details
In all the animations, the red object (the target) started
to move first, describing an arc from the center of thescreen that ended in one of the four corners of thescreen (the example shown in Figure 1 ends in the bottom left corner; during the experiment, an equal number of trials with end point in each corner were shown).
The blue object (the chaser) started 0.93 sec after thered object, and in half the trials it tried to catch it(chasing trials), in the other half of the trials it moved
in the opposite direction from the red object (control

1702

Journal of Cognitive Neuroscience

trials). When trying to catch the red object, the blue
object either simply followed the target’s trajectory or itwent directly to the end point of the target’s movement,
as if it had known the end point of the red object’s
trajectory. In half of the chasing trials the blue object
caught the red object, in the other half of the trials it
missed it by 68.
Image Acquisition
A Shimadzu-Marconi ECLIPSE scanner (Shimadzu, Japan; Marconi, London, UK), operating at 1.5 T was used
to acquire both T1-weighted anatomical images andgradient-echo echo-planar T2*-weighted MRI images
with blood oxygenation level dependent (BOLD) contrast. The scanning sequence was an echo-planar imaging sequence with repetition time of 4500 msec, anecho time of 45 msec, a flip angle of 908, a field of viewof 256  256 mm and a matrix size of 64  64. Eachfunctional image comprised 36 axial slices, with athickness of 3 mm and a 1-mm interval between slices,and was positioned to cover the whole brain. For eachsubject, 260 functional images were acquired in one
session of 19.5 min, including five subsequently discarded ‘‘dummy’’ volumes at the start of the session toallow for T1 equilibration effects. A high-resolution
structural MR image was acquired for each subject witha T1-weighted sequence.
fMRI Data Analysis
Statistical analysis was carried out using the General
Linear Model framework (Friston et al., 1995) implemented in the SPM2 software package (Wellcome Department of Imaging Neuroscience, www.fil.ion.ucl.
ac.uk/spm). To correct for subject motion, the functional
images were realigned with the first functional image andresliced (Friston et al, 1995). Images were then normalized into a standard EPI T2* template with a resampled
voxel size of 2 mm3 (Friston et al., 1995). To enhance thesignal-to-noise ratio and enable intersubject functional
anatomical comparison, the images were smoothed byconvolution with a 6-mm full width at half maximum(FWHM) gaussian kernel. A high-pass filter (using a cutoffof 128 sec) and a correction for temporal autocorrelationin the data (AR 1 + white noise) were applied to accommodate serial correlations.
A mixed-effects analysis was adopted, using a twostage procedure. First, a fixed-effects analysis was applied separately to the preprocessed data of each subject
using the general linear model implemented in SPM2.
This consists of fitting the data with a linear combinationof regressors in a design matrix, to produce 3-D maps ofparameter estimates. These parameter estimates represent the contribution of a particular regressor to thedata, and can be transformed to percent BOLD signal
change with respect to the average global signal acrossVolume 16, Number 10

conditions and voxels. The design matrix used in thisexperiment consisted of 32 regressors. There were 16
conditions in the experiment, engendered by crossing
the following factors: (i) following or predicting strategyof the chaser, (ii) successful or unsuccessful outcome,(iii) task (strategy categorization or outcome categorization), or (iv) animations with chasing or controls. Two
regressors were created for each condition in the following way: The duration from stimulus presentation
onset to the subject’s button press was modeled by aseries of delta (‘‘stick’’) functions, then convolved by acanonical hemodynamic response function (HRF) as
implemented in SPM2 (first regressor for this condition) and its first temporal derivative (second regressor
for the condition). To correct for movement-related
artifacts not eliminated during realignment, differential
realignment parameters were modeled as additional
regressors of no interest.
For each subject, linear contrasts of parameter estimates were used to assess the effects of following factors: (i) predictive strategy versus following strategy of
the chasing object, controlling for nonspecific object motion effects (interaction between predictive vs. following
strategy and chase vs. control trials) and (ii) effect of task
on the previous comparison (interaction between predictive vs. following strategy, chase vs. control trials, andstrategy task vs. outcome task). As an additional measureof the effect of task on the observation of chasing strategy, we assessed separately the interaction between predictive versus following strategy and chase versus control
trials in the strategy task and in the outcome task. We
then masked the results of this contrast in the strategy
task with the results of the same contrast in the outcome
task with an exclusive masking procedure implemented
in SPM2. The threshold of the mask used in the figures
was the standard p = .05; similar results were obtainedwith all thresholds below p = .15.
One-sample t tests were performed on the above
contrast images to give second-level or random effects

SPMs, after further smoothing the weighted maps by
convolution with an 8-mm FWHM gaussian kernel toaccount for anatomical differences across subjects.
On the basis of published work on perception of
biological motion, abstract moving agents, and theoryof-mind tasks reported in the Introduction, we expectedchanges in activation to occur in the cortex surrounding
the posterior part of the superior temporal sulcus and in
the superior temporal gyrus of both hemispheres. We
therefore defined anatomical search regions that weused to perform a correction for multiple corrections
across all voxels of each of these search regions (Poline,
Worsley, Evans, & Friston, 1997). The voxels corresponding to these search regions were defined on thebasis of the subjects’ structural scans and an anatomicalatlas of the human brain (Duvernoy, 1999), and weredrawn using MRIcro software (Rorden & Brett, 2000)(www.cla.sc.edu/psyc/faculty/rorden/mricro.html). To
test specifically for activation in the medial prefrontal
cortex, we defined search regions that encompass allstudies of mentalizing cited in a recent review (Frith &Frith, 2003). Clusters of more than four voxels in extent
and with a p value of <.05 after correction for multiple
comparisons across all voxels in the search region wereconsidered to be significantly activated and are reported
in Table 1 and Figures 2 and 3. The search region in theright posterior part of the superior temporal sulcus
extended from 68 to 52 mm in the x dimension,from 66 to 34 mm in the y dimension, and from
+2 to +30 mm in the z dimension. The region in theright superior temporal sulcus extended from +50 to+70 mm in x, from 65 to 37 mm in y, and from 2 to
+29 mm in z. Volumes were 6049 mm3 (left) and
7284 mm3 (right). The region in the left superior temporal gyrus extended from 71 to 46 mm in x, 54
to +23 mm in y, and 31 to +21 mm in z. The region
in the right superior temporal gyrus extended from + 42
to +72 mm in x, 52 to +25 mm in y, and 34 to +31 mm
in z. Volumes were 33,935 mm3 (left) and 45,896 mm3Table 1. Brain Activation Data, Voxelwise Analysis
Structure

Coordinates

Size (mm3)Z scorePredict > Follow strategy * Chase versus Control
R superior temporal gyrus

48

44

12

600

4.48

L middle temporal gyrus

60

56

4

176

3.83

L superior temporal gyrus

56

30

4

584

3.58

4

936

4.2

784

4.2

Predict > Follow strategy * Chase versus Control, Strategy task only
L superior temporal gyrus

54

34

Predict > Follow strategy * Chase versus Control, Strategy task masked by Outcome task
L superior temporal gyrus

54

34

4

Significant clusters in the comparisons of interest, surviving a threshold of p < .05, corrected for multiple comparisons across voxels in the superior
temporal sulcus or gyrus (see Methods).

Schultz et al.

1703

(right). The search regions in the left medial prefrontal
cortex extended from 18 to +0 in the x dimension,from +20 to +60 in the y dimension, and from 2 to 52
in the z dimension. The region in the right medial
prefrontal cortex extended from +0 to +16 in the x dimension, from +14 to +60 in the y dimension, and from+4 to +54 in the z dimension. Volumes were 12,112 mm3(left) and 12,448 mm3 (right).
Peristimulus time courses were obtained by extracting data from the peaks of activation in the superior
temporal gyrus of both hemispheres, isolated in theinteraction between predict versus follow strategy andchase versus control. Coordinates of the activation
peaks were 54 34 4 and 48 44 12. Data from alltime points were tested for normality by comparison toa normal distribution with one-sample Kolmogorov–
Smirnov tests (all p > .1). Two-way ANOVAs were used
to assess differences on activation over time between
trials with predictive and outcome strategy for each task
and each hemisphere.
Image Used for Display
The mean image used for display in the figures wascalculated by averaging the 12 subjects’ structural images that were previously coregistered with the meanfunctional image of the same subject and normalized tothe standard Montreal Neurological Institute (MNI)
space. Anatomical structures were identified with brain
atlases by Duvernoy (1999).
Acknowledgments
The authors thank the radiographers at the ATR Brain Activity
Imaging Center and the volunteers who participated in thisstudy. Johannes Schultz would like to thank Daniel Wolpert
for arranging the contact to the ATR laboratories. Johannes
Schultz and Chris D. Frith are supported by the Wellcome
Trust. This study was supported by the James S. McDonnell
Foundation. Its contents are solely the responsibility of theauthors and do not necessarily represent the official views of
the James S. McDonnell Foundation.
Reprint requests should be sent to Johannes Schultz, Wellcome
Department of Imaging Neuroscience, 12 Queen Square,
London WC1N 3BG, UK, or via e-mail: j.schultz@fil.ion.ucl.
ac.uk.
The data reported in this experiment have been deposited in
the fMRI Data Center (http://www.fmridc.org). The accession
number is 2-2004-116RA.

REFERENCES
Bassili, J. N. (1976). Temporal and spatial contingencies in theperception of social events. Journal of Personality andSocial Psychology, 33, 680–685.
Blakemore, S. J., Boyer, P., Pachot-Clouard, M., Meltzoff, A.,
Segebarth, C., & Decety, J. (2003). The detection ofcontingency and animacy from simple animations in thehuman brain. Cerebral Cortex, 13, 837–844.

1704

Journal of Cognitive Neuroscience

Blakemore, S. J., Fonlupt, P., Pachot-Clouard, M., Darmon, C.,
Boyer, P., Meltzoff, A. N., Segebart, C., & Decety, J. (2001).
How the brain perceives causality: An event-related fMRI
study. NeuroReport, 12, 3741–3746.
Blythe, P. W., Todd, P. M., & Miller, G. F. (1999). How
motion reveals intention: Categorising social interactions.
In G. Gigerenzer & P. M. Todd (Eds.), Simple heuristics
that make us smart (pp. 257–285). Oxford: Oxford
University Press.
Bonda, E., Petrides, M., Ostry, D., & Evans, A. (1996). Specificinvolvement of human parietal systems and the amygdalain the perception of biological motion. Journal of
Neuroscience, 16, 3737–3744.
Brunet, E., Sarfati, Y., Hardy-Bayle, M.-C., & Decety, J. (2000). A
PET investigation of the attribution of mental intentions witha nonverbal task. Neuroimage, 11, 157–166.
Campbell, R., MacSweeney, M., Surguladze, S., Calvert, G.,
McGuire, P., Suckling, J., Brammer, M. J., & David, A. S.
(2001). Cortical substrates for the perception of faceactions: An fMRI study of the specificity of activation forseen speech and for meaningless lower-face acts (gurning).
Brain Research Cognitive Brain Research, 12, 233–243.
Castelli, F., Happe, F., Frith, U., & Frith, C. (2000). Movement
and mind: A functional imaging study of perception andinterpretation of complex intentional movement patterns.
Neuroimage, 12, 314–325.
Csibra, G. (2003). Teleological and referential understanding ofaction in infancy. Philosophical Transactions of the Royal
Society of London. Series B, Biological Sciences, 358,
447–458.
Downar, J., Crawley, A. P., Mikulis, D. J., & Davis, K. D. (2000).
A multimodal cortical network for the detection of changes
in the sensory environment. Nature Neuroscience, 3,
277–283.
Duvernoy, H. M. (1999). The human brain. Vienna: Springer
Verlag.
Fletcher, P. C., Happe, F., Frith, U., Baker, S. C., Dolan, R. J.,
Frackowiak, R. S., & Frith, C. D. (1995). Other minds in thebrain: A functional imaging study of ‘theory of mind’ in story
comprehension. Cognition, 44, 283–296.
Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. B.,
Frith, C. D., & Frackowiak, R. S. (1995). Statistical parametricmapping in functional imaging: A general linear approach.
Human Brain Mapping, 2, 189–210.
Frith, C. D., & Frith, U. (1999). Interacting minds—A biologicalbasis. Science, 286, 1692–1695.
Frith, U., & Frith, C. D. (2003). Development andneurophysiology of mentalizing. Philosophical Transactionsof the Royal Society of London. Series B, Biological
Sciences, 358, 459–473.
Gallagher, H. L., Happe, F., Brunswick, N., Fletcher, P. C.,
Frith, U., & Frith, C. D. (2000). Reading the mind in cartoonsand stories: An fMRI study of ‘‘theory of mind’’ in verbal andnonverbal tasks. Neuropsychologia, 38, 11–21.
Giese, M. A., & Poggio, T. (2003). Neural mechanisms for therecognition of biological movements. Nature Reviews
Neuroscience, 4, 179–192.
Goel, V., Grafman, J. N. S., & Hallett, M. (1995). Modellingother minds. NeuroReport, 6, 1741–1746.
Grezes, J., Armony, J. L., Rowe, J., & Passingham, R. E. (2003).
Activations related to ‘‘mirror’’ and ‘‘canonical’’ neuronesin the human brain: An fMRI study. Neuroimage, 18,
928–937.
Grezes, J., & Costes, N. (1998). Top-down effect of strategy
on the perception of biological motion: A PET investigation.
Cognitive Neuropsychology, 15, 582.
Grossman, E., Donnelly, M., Price, R., Pickens, D., Morgan, V.,
Neighbor, G., & Blake, R. (2000). Brain areas involved inVolume 16, Number 10

perception of biological motion. Journal of Cognitive
Neuroscience, 12, 711–720.
Heider, F., & Simmel, M. (1944). An experimental study ofapparent behaviour. American Journal of Psychology, 57,
243–259.
Hoffman, E. A., & Haxby, J. V. (2000). Distinct representationsof gaze and identity in the distributed human neuralsystem for face perception. Nature Neuroscience, 3,
80–84.
Iacoboni, M., Koski, L. M., Brass, M., Bekkering, H.,
Woods, R. P., Dubeau, M. C., Mazziotta, J. C., & Rizzolatti, G.
(2001). Reafferent copies of imitated actions in the rightsuperior temporal cortex. Proceedings of the National
Academy of Sciences, U.S.A., 98, 13995–13999.
Johansson, G. (1973). Visual perception of biological motion
and a model for its analysis. Perception and Psychophysics,
14, 201–211.
Johnson, S. C. (2003). Detecting agents. PhilosophicalTransactions of the Royal Society of London. Series B,
Biological Sciences, 358, 549–559.
Klin, A. (2000). Attributing social meaning to ambiguous visual
stimuli in higher functioning autism and Asperger
syndrome: The social attribution task. Journal of Child
Psychology and Psychiatry, 41, 846.
Narumoto, J., Okada, T., Sadato, N., Fukui, K., & Yonekura, Y.
(2001). Attention to emotion modulates fMRI activity inhuman right superior temporal sulcus. Brain Research.
Cognitive Brain Research, 12, 225–231.
Neri, P., Morrone, M. C., & Burr, D. C. (1998). Seeing biologicalmotion. Nature, 395, 894–896.
Opfer, J. E. (2002). Identifying living and sentient kinds fromdynamic information: The case of goal-directed versus
aimless autonomous movement in conceptual change.
Cognition, 86, 97–122.
Perrett, D. I., Smith, P. A., Mistlin, A. J., Chitty, A. J., Head, A. S.,
Potter, D. D., Broennimann, R., Milner, A. D., & Jeeves, M. A.
(1985). Visual analysis of body movements by neurones inthe temporal cortex of the macaque monkey: A
preliminary report. Behavioural Brain Research, 16,
153–170.

Poline, J. B., Worsley, K. J., Evans, A. C., & Friston, K. J.
(1997). Combining spatial extent and peak intensity totest for activations in functional imaging. Neuroimage, 5,
83–96.
Puce, A., Allison, T., Bentin, S., Gore, J. C., & McCarthy, G.
(1998). Temporal cortex activation in humans viewing eye
and mouth movements. Journal of Neuroscience, 18,
2188–2199.
Puce, A., & Perrett, D. (2003). Electrophysiology and brain
imaging of biological motion. Philosophical Transactions of
the Royal Society of London. Series B, Biological Sciences,
358, 435–445.
Rorden, C., & Brett, M. (2000). Stereotaxic display of brain
lesions. Behavioural Neurology, 12, 191–200.
Saxe, R., & Kanwisher, N. (2003). People thinking about
thinking people. The role of the temporo-parietal junctionin ‘‘theory of mind’’. Neuroimage, 19, 1835–1842.
Scholl, B. J., & Tremoulet, P. D. (2000). Perceptual causality
and animacy. Trends in Cognitive Sciences, 4, 299–309.
Schultz, R. T., Grelotti, D. J., Klin, A., Kleinman, J.,
Van der, G. C., Marois, R., Skudlarski, P. (2003). The role ofthe fusiform face area in social cognition: Implications forthe pathobiology of autism. Philosophical Transactions of
the Royal Society of London. Series B, Biological Sciences,
358, 415–427.
Vogeley, K., Bussfeld, P., Newen, A., Hermann, S., Happe, F.,
Falkai, P., Maier, W., Shah, N. J., Fink, G. R., & Zilles, K.
(2001). Mind reading: Neural mechanisms of theory of mind
and self-perspective. Neuroimage, 14, 181.
Wicker, B., Michel, F., Henaff, M., & Decety, J. (1998). Brainregions involved in the perception of gaze: A PET study.
Neuroimage, 8, 221–227.
Winstein, C. J., Grafton, S. T., & Pohl, P. S. (1997). Motor taskdifficulty and brain activity: Investigation of goal-directed
reciprocal aiming using positron emission tomography.
Journal of Neurophysiology, 77, 1581–1594.
Winston, J. S., Strange, B. A., O’Doherty, J., & Dolan, R. J.
(2002). Automatic and intentional brain responses during
evaluation of trustworthiness of faces. Nature Neuroscience,
5, 277–283.

Schultz et al.

1705