# Reinforcement effects in anticipatory smooth eye movements

Jean-Bernard Damasse, Laurent U Perrinet, Laurent Madelain, Anna Montagnini

<figure><img src="image_1.png" /></figure>

## To cite this version:

Jean-Bernard Damasse, Laurent U Perrinet, Laurent Madelain, Anna Montagnini. Reinforcement effects in anticipatory smooth eye movements. Journal of Vision, 2018, 18 (11), 10.1167/18.11.14. hal-01901640

HAL Id: hal-01901640

https://amu.hal.science/hal-01901640

Submitted on 23 Oct 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
Reinforcement effects in anticipatory smooth eye movements

Jean-Bernard Damasse, Laurent U Perrinet, Laurent Madelain, Anna Montagnini

► To cite this version:

Jean-Bernard Damasse, Laurent U Perrinet, Laurent Madelain, Anna Montagnini. Reinforcement effects in anticipatory smooth eye movements. Journal of Vision, Association for Research in Vision and Ophthalmology, 2018, 18 (11), <10.1167/18.11.14>. <hal-01901640>

HAL Id: hal-01901640

https://hal-amu.archives-ouvertes.fr/hal-01901640

Submitted on 23 Oct 2018

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire **HAL**, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
# Reinforcement effects in anticipatory smooth eye movements

**Jean-Bernard Damasse**
Aix Marseille Université, CNRS, Institut de Neurosciences de la Timone UMR 7289, Marseille, France

**Laurent U. Perrinet**
Aix Marseille Université, CNRS, Institut de Neurosciences de la Timone UMR 7289, Marseille, France

**Laurent Madelain**
University of Lille Nord de France, CNRS, SCALAB UMR 9193, Lille, France

**Anna Montagnini**
Aix Marseille Université, CNRS, Institut de Neurosciences de la Timone UMR 7289, Marseille, France

When predictive information about target motion is available, anticipatory smooth pursuit eye movements (aSPEM) are consistently generated before target appearance, thereby reducing the typical sensorimotor delay between target motion onset and *foveation*. By manipulating the probability for target motion direction, we were able to bias the direction and mean velocity of aSPEM. This suggests that motion-direction expectancy has a strong effect on the initiation of anticipatory movements. To further understand the nature of anticipatory smooth eye movements, we investigated different effects of *reinforcement* on aSPEM. In a first experiment, the reinforcement was contingent to a particular anticipatory behavior. A monetary reward was associated to a criterion-matching anticipatory velocity as estimated online during the gap before target motion onset. Our results showed a small but significant effect of behavior-contingent monetary reward on aSPEM. In a second experiment, the proportion of rewarded trials was manipulated across motion directions (right vs. left) independently from participants' behavior. Our results indicate that a bias in expected reward does not systematically affect anticipatory eye movements. Overall, these findings strengthen the notion that anticipatory eye movements can be considered as an operant behavior (similar to visually guided ones), whereas the expectancy for a noncontingent reward cannot efficiently bias them.

## Introduction

Natural environments offer myriad potentially relevant visual targets at the same time. Target selection is, therefore, one of the main functions of voluntary eye

movements. To do so, humans and primates, in general, use saccades to quickly foveate objects of interest. In the case of smoothly moving objects, the oculomotor system can also elicit smooth pursuit eye movements that stabilize the retinal image of the selected target on the fovea, minimizing the motion blur that would otherwise alter the perception of the moving object.

Historically, the pursuit system has been regarded as a servo system in which the delayed retinal slip first drives smooth eye movements throughout an open-loop phase (Robinson, 1986), i.e., before the visual feedback has time to affect the ongoing movement. Then, during a closed-loop phase, information about the actual eye movement is integrated into the oculomotor controller through a positive feedback loop, virtually enabling the eye movement to exactly match the target movement (Lisberger, Morris, & Tychsen, 1987; Tychsen & Lisberger, 1986). However, several observations suggest that such simple servo-mechanisms do not capture the whole richness of smooth pursuit (e.g., G. R. Barnes, 2008; Bogadhi, Montagnini, & Masson, 2013; Madelain & Krauzlis, 2003a, 2003b; Orban de Xivry et al., 2013; Perrinet, Adams, & Friston, 2014). In particular, it has been established that smooth pursuit eye movements can be strongly modulated by perceptual and cognitive factors (for reviews, see G. R. Barnes, 2008; Sperling & Montagnini, 2011), such as visual attention (Berman et al., 1999; Chen, Holzman, & Nakayama, 2002; Culham et al., 1998; Ferrera & Lisberger, 1995; Keller & Khan, 1986; Khurana & Kowler, 1987; Souto & Kerzel, 2008), object-related processing (Ilg & Churan, 2004; Stone, Beutter, & Lorenceau, 2000) or illusory motion perception (Beutter & Stone, 2000; Lorenceau, 2012;

Citation: Damasse, J.-B., Perrinet, L. U., Madelain, L., & Montagnini, A. (2018). Reinforcement effects in anticipatory smooth eye movements. *Journal of Vision*, *18*(11):14, 1–18, https://doi.org/10.1167/18.11.14.
Madelain & Krauzlis, 2003b; Montagnini, Mamassian, Perrinet, Castet, & Masson, 2007; Stone et al., 2000; Spering, Kerzel, Braun, Hawken, & Gegenfurtner, 2005; Wallace, Stone, & Masson, 2005). Importantly, and contrary to the conventional notion that some visual motion input (or, to a lesser extent, some motion input from another sensory modality) is necessary to initiate smooth pursuit, the predictability of the target's motion can lead to anticipatory initiation of pursuit before target motion onset or to the maintenance of smooth eye movements during transient occlusions of the moving target (Badler & Heinen, 2006; G. R. Barnes & Asselman, 1991; Becker & Fuchs, 1985; Bennett & Barnes, 2006; Collewijn & Tamminga, 1984; Freyberg & Ilg, 2008; Kowler, 1989; Kowler, Martins & Pavel, 1984; Kowler & Steinman, 1979a, 1979b; Landelle, Montagnini, Madelain, & Danion, 2016; Madelain & Krauzlis, 2003a; Orban de Xivry, Missal, & Lefèvre, 2008; Wells & Barnes, 1999).

It has been recently proposed that oculomotor control strongly depends on reinforcement contingencies such that eye movements may adapt to environmental variable constraints (see Madelain, Paeye, & Darcheville, 2011, for review). There are numerous studies on reinforcement and saccadic eye movements, but only a handful of studies probed the sensitivity of smooth pursuit to reinforcement contingencies. In newborn infants (1–7 days), Darcheville, Madelain, Buquet, Charlier, and Miossec (1999) found that smooth eye movements can be enhanced using an auditory reinforcer contingent to the eye velocity. Schütz, Lossin, and Gegenfurtner (2015) showed that, in presence of two moving textures, pursuit direction choice is mainly determined by the target's visual salience at initiation, but 300–400 ms after motion onset, pursuit steered toward the rewarded direction, and the salience effects gradually disappeared. On the other hand, Joshua and Lisberger (2012) reported an influence of precued reward contingencies already on vector average pursuit initiation in monkeys. Finally, using a single moving target, Brielmann and Spering (2015) have shown that human participants' smooth pursuit is affected by reward expectation across the whole time course, whether the stimulus–reward association is precued or not. Moreover, reinforcement can also affect pursuit in the absence of a visual target. During the transient blanking of a moving target, human subjects can still elicit smooth tracking but with a natural reduction by 40%–60% of the normal eye velocity (Becker & Fuchs, 1985). Madelain and Krauzlis (2003a) successfully trained participants to maintain high gain pursuit while the visual moving target was transiently occluded with contingencies such that a reward (signaled by a contingent continuous auditory tone during pursuit) was given, depending on

whether the eye velocity accurately matched the hidden target velocity.

Here we investigate the effects of learning on a particular phase of human smooth pursuit, namely the anticipatory smooth pursuit eye movements (aSPEM) that can be observed ahead of target motion onset under particular conditions. Indeed, when temporal or spatial cues before target motion onset make the forthcoming motion properties predictable to some degree, anticipatory smooth tracking movements are generated. Several studies have previously looked at the influence of cognitive factors on anticipatory smooth pursuit (G. R. Barnes & Asselman, 1991; de Hemp tinne, Lefèvre, & Missal, 2008; Heinen, Badler, & Ting, 2005; Kowler, 1989; Kowler et al., 1984; Kowler & Steinman, 1979a, 1979b; Van den Berg, 1988; Wells & Barnes, 1999). It is now commonly accepted that anticipatory pursuit is strongly impaired if the expected motor response has not been previously experienced (G. Barnes, Grealy, & Collins, 1997) and that the efficiency in eliciting anticipation can vary dramatically depending on the type of cues (Kowler, Aitkin, Ross, Santos, & Zhao, 2014). Regularities in the target motion properties also strongly affect anticipatory smooth pursuit (Kowler et al., 1984; Kowler & Stein man, 1979a, 1979b; Montagnini, Souto, & Masson, 2010; Santos & Kowler, 2017). However, none of these studies has probed the extent of the effects of reinforcement contingencies on anticipatory smooth eye movements.

In this paper, we used eye-movement recordings as behavioral measures in an eye velocity–contingent operant conditioning paradigm specifically designed to target aSPEM. We report effects of monetary rein- forcement contingencies on anticipatory smooth eye velocity and confirmed these with a yoked-control procedure (Wasserman, 2010). We also aimed at further clarifying the role of operant versus nonoperant components of reward-related behavioral changes by implementing an associative learning procedure (Pav- lov, 1927) and comparing it to results from the operant conditioning paradigm. Overall, our results suggest that anticipatory eye movements are affected by reinforcement contingencies in addition to other cognitive and predictive processes.

## Methods

### Subjects

Nineteen human subjects (10 females and nine males, aged 23–43) participated in Experiment 1; two of them were excluded from further analyses due to over-noisy eye-tracking recordings. The remaining 17 participants
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

<figure><img src="image_5.png" /></figure>

Figure 1. Presentation of the experimental conditions. (A) Experiment 1, motion-direction bias task (baseline). After a random-duration fixation followed by a 300-ms gap interval, a small ring-shaped target moved from the center of the screen toward the left or the right side with constant velocity for 750 ms. The probability *P* of rightward motion direction was manipulated across experimental blocks. (B) Experiment 2, operant-conditioning pursuit tasks. Similar to Experiment 1, but the color of the moving target was dependent on the comparison of the online estimation of the eye velocity during the critical comparison window (between –100 ms and motion onset, see schematic illustration in the figure inset, which also summarizes the different rules for reward delivery) with an adaptive criterion velocity. The color of the target also informed about the money gain or loss (green target = gain, red target = loss) for the current trial. (C) Experiment 3, associative learning pursuit task. Similar to Experiment 1, but a given proportion of trials in each direction was associated with a monetary gain (green target, rewarded trial) regardless of the participant's oculomotor behavior, and the remaining fraction of trials (white target) was neither rewarded nor punished. In this experiment, the target motion direction was never biased.

were also assigned to three other experiments. Some of them participated in several experiments, depending on their availability. Seven participants were involved in Experiment 2, four in the yoked control condition, and six in Experiment 3. Two of the participants were authors of the study for Experiment 1 and one for Experiment 3, whereas all other participants were naïve as to the experimental conditions and goals of the studies. Naïve subjects received a fixed amount of money

for their participation. However, for Experiment 2 (both for the main condition and the yoked control), they were instructed at the beginning of the experiment that the sum they would receive would depend on their performance. All subjects were healthy and had normal or corrected-to-normal vision and had no relevant medical or psychiatric history. The experiments were conducted in accordance with the ethical regulations of the Centre National de la Recherche Scientifique after
<figure><img src="image_6.png" /><figcaption>Figure 2. Average horizontal smooth eye velocity across trials for rightward and leftward target motion direction (solid and dashed curves, respectively) and for different direction-bias sessions (color-coded; see figure legend). The solid vertical black line marks target motion onset. The dashed vertical lines mark the limits of the anticipation window, which was used to estimate the mean anticipatory velocity in each trial.</figcaption></figure>

obtaining the approval of the ethical committee of Aix-Marseille Université (approval no. 2014-12-3-05) for behavioral noninvasive research and in accordance with the Declaration of Helsinki. Written informed consent was obtained from all participants.

## Apparatus

Stimuli were presented on a 21-in. CRT monitor with a refresh rate of 100 Hz. Stimuli were generated using the Psychophysics Toolbox extension for MATLAB (Brainard, 1997; Pelli, 1997) running on a Mac Pro (first-generation) computer and displayed at a viewing distance of 57 cm against a gray background. The luminance of the gray background was 42 cd/m². To minimize measurement errors, the subject's head movements were restrained using a chin and forehead rest so that the eyes in primary gaze position were directed toward the center of the screen. Monocular eye movements were measured continuously with an eye-tracking system (Eyelink 1000, SR Research Ltd., sampled at 1,000 Hz), and data were transferred, stored, and analyzed off-line using programs written in MATLAB or Ipython notebooks.

## Data analysis

Recorded horizontal and vertical gaze positions were low-pass filtered using a Butterworth (acausal) filter of

order 2 with a 30-Hz cutoff frequency and then were numerically differentiated to obtain velocity measures. We used an automatic conjoint acceleration and velocity threshold method to detect saccades (Krauzlis & Miles, 1996) and further inspected all individual traces visually to exclude aberrant trials. Mean eye velocity traces were obtained by trial averaging. For this purpose, saccades were excluded from the data by tagging them as “nonvalid,” and averaging was computed over valid data points. We evaluated the effects of the experimental manipulations on anticipatory velocity at the individual level by comparing the mean eye velocity during a temporal anticipation window between -50 and +50 ms around target motion onset (as highlighted in Figure 2), using individual one-way ANOVAs (with the probability bias or reinforcement type as three-level single factors). We also computed all post hoc pair-wise comparisons using Tukey's honestly significant difference (HSD) test. Even though the normality assumption of our data was fairly justified given the sample's size (always more than 60 trials per condition), we also computed nonparametric statistics (Kruskal-Wallis test) for comparison, and results were similar. All tests and analyses have been realized using Python 3.5 with the Numpy and Scipy libraries.

## Experiment 1: Motion-direction bias task (baseline)

The visual stimulus used in Experiment 1 was a white ring (0.30° outer diameter and 0.23° inner diameter) with a luminance of 102 cd/m² that moved horizontally on a gray background. Each trial started with a central fixation point displayed for a random duration drawn from a uniform distribution ranging between 300 and 600 ms. Then a fixed-duration 300-ms gap occurred between the offset of the fixation point and the onset of the moving target, which was presented at the fixation location and immediately started moving horizontally at a constant speed of 7.7°/s either to the right or the left for 1,000 ms. Across experimental sessions, the probability *P* of rightward trials was manipulated in order to create a direction bias and favor the buildup of direction expectancy (See Figure 1A). The experiment included three sessions. The first session had *P* = 0.50, hence about 200 rightward trials out of 400 trials; the second one had *P* = 0.75 (about 300 rightward trials out of 400 trials); the last one *P* = 0.90 out of a total of either 400 (360 rightward) or 600 trials (540 rightward; respectively, nine and 10 participants). The rationale for having a larger number of trials in the *P* = 0.90 session was to collect enough trials for the least frequent direction, namely the left one, but after running the first participants, we realized that the
detailed analysis of the least-frequent trials did not provide, per se, a major advantage for the present study because the anticipatory behavior was clearly observable with the standard 400 trial-sessions. As we wanted to test a condition without any direction uncertainty, four participants also completed an additional session of 400 rightward trials only ($P = 1.00$). With the exception of the $P = 1$ condition, target motion direction was pseudorandomized across trials once for each session type, such that all participants were presented with the exact same sequence of randomly alternating directions. Participants were instructed to track the target as accurately as possible.

## Experiment 2: Operant conditioning of aSPEM

The visual stimuli used in Experiment 2 were two rings (0.30° outer diameter and 0.23° inner diameter), one was green (luminance of 90 cd/m²) and the other red (luminance of 22 cd/m²). For each reinforcement condition (see below), there were three sessions in which the target direction bias and the timeline of each trial was very similar to the baseline experiment, but here we applied an eye velocity-contingent reinforcement schedule based on anticipatory smooth pursuit. We defined a 100-ms critical time window for anticipatory pursuit starting 100 ms before motion onset. Note that this window (which we refer to as a *critical comparison window*) is slightly different from the previously defined *anticipation window* (between -50 and 50 ms) as the comparison between the eye velocity and the criterion has to occur strictly before target motion onset in order to enforce a causal association with the reward cue (coincident with the target's color). The mean online eye-velocity estimate in the critical comparison window was compared to a velocity criterion. The criterion velocity was defined as the median of the anticipatory velocity estimated over the 20 previous trials. The criterion was, therefore, constantly evolving and depended on the participant's anticipatory behavior during the recent past. However, depending on the actual experimental condition, the criterion was further constrained to only increase or decrease (see below). For the first 20 trials of each experimental session, the criterion was defined as the median velocity of each participant during the corresponding (bias-matched) baseline session. We implemented two reinforcement conditions that were administered to each of the seven participants in separate sessions (see details on the order of presentation below):

* The first condition was the “anticipation booster,” whereby we rewarded increased anticipatory velocities. If the online anticipatory eye velocity estimate was higher than the criterion, then the

green ring stimulus was displayed as a moving pursuit target. If it was smaller than the criterion, then a red ring target was displayed (see Figure 1B). The instructions given to participants were to accurately track the moving target regardless of its color and that participants were going to obtain 0.30 euros for each trial with a green ring and lose 0.10 euros for each trial with a red ring. Every 20 trials, a text message appeared on the screen stating whether the cumulative earnings were increasing or decreasing. Importantly, the velocity criterion was, by construction, only allowed to increase or remain constant but could never decrease in this condition.

* The second condition was the “anticipation braker” in which decreased anticipatory velocities were rewarded. When the online anticipatory velocity estimate was smaller than the criterion in a given trial, we gave a reward (green moving target and +0.30 euros), and if it was higher, we gave a punishment (red moving target and -0.10 euros). In this experimental condition, the criterion could either decrease or remain constant but could never increase.

It should be noted that every saccade during the critical comparison window (defined on the basis of online estimated mean velocity exceeding 70°/s) was followed by a red target (thus, “punished”) whether in the booster or braker condition.

We further tested a third condition, namely a yoked-control procedure, whereby a reward or punishment was provided on each trial regardless of the actual smooth pursuit performance. In practice, the yoked procedure was used to isolate any effects generally due to rewards but not to the contingency between anticipatory behaviors and rewards or punishments. Similarly to the baseline, we used the three sessions with a probabilistic bias for the target motion direction. All of the four participants of the yoked-control group had already participated in Experiment 1 and two of them also in the reinforcement conditions. Following the standard procedure for yoked control, the rate of reward for the participants of the yoked condition was randomly selected among the reward rates estimated for four other randomly matched participants of the reinforcement conditions. The instructions given to the participants were identical as in the booster/braker conditions despite the fact that, this time, the outcome was not causally related to the anticipatory behavior. Again, saccades occurring in the critical comparison window were punished. To closely stick to the rate of reward given in the matched reinforcement condition regardless of the individual number of anticipatory saccades, we readjusted in an online fashion the number of rewarded and punished trials of the yoked condition accounting for the saccade-based punished trials.
<figure><img src="image_7.png" /><figcaption>Figure 3. (a) Histogram and trial average (vertical line) of anticipatory smooth eye velocity for each direction-bias session for participant s6. (b) Boxplots of anticipatory eye velocity for the whole group of participants of Experiment 1 in all direction-bias sessions. Individual mean velocities and 95% confidence interval are also presented with black symbols connected by thin lines. The individual data points' abscissae are randomly jittered on the x-axis in order to avoid the superposition of the symbols.</figcaption></figure>

## Experiment 3: Associative learning and aSPEM

To investigate in more detail the nature of reward-related effects on anticipatory smooth pursuit and, in particular, to clarify the role of the operant-conditioning procedure, we implemented an associative learning procedure in which a specific motion direction was associated with the reward. Participants were presented with three 400-trial sessions of the standard pursuit experiment without any direction bias, namely with $P = 0.50$ for rightward and leftward trials. We introduced a bias in the rate of rewarded trials for each target motion direction: In the first session, we pseudorandomly rewarded a proportion $P_{rew} = 0.50$ of rightward trials (and $1 - P_{rew} = 0.50$ of leftward trials); in the second session, we pseudorandomly rewarded $P_{rew} = 0.75$ of rightward trials (and $1 - P_{rew} = 0.25$ of leftward trials), and in the third session, we pseudorandomly rewarded $P_{rew} = 0.90$ of rightward trials (and $1 - P_{rew} = 0.10$ of leftward trials; see Figure 1C). The instructions were the same as in Experiment 2.

### Experimental conditions: Order of presentation

The order of presentation of the different direction-bias conditions was fixed for Experiment 1. All participants were first tested with $P = 0.5$, then $P = 0.75$, and finally $P = 0.90$. Out of the seven participants in Experiment 2, four (s5, s6, s7, and s8) were first presented with the booster condition followed by the braker condition. Three participants (s16, s17, s18) experienced the braker condition first, followed by the booster condition. In Experiment 2, the order of

direction bias was fixed for most participants (first $P = 0.5$, then $P = 0.75$, then $P = 0.9$, and finally $P = 1$, when this last condition was implemented) with the exception of participants s7 and s8 who underwent a different order of presentation: first $P = 0.90$, then $P = 0.5$, and finally $P = 0.75$ for s7; first $P = 0.75$, then $P = 0.5$, and finally $P = 0.90$ for s8. Finally, for Experiment 3, the order of presentation of the different conditions was fixed; namely participants were first provided with $P_{rew} = 0.50$ of rightward rewarded trials, then $P_{rew} = 0.75$ and last $P_{rew} = 0.90$.

## Results

### Experiment 1: Direction-bias task (baseline)

In this experiment, we introduced an increasing proportion of rightward trials throughout the sessions. Figure 2 shows the average smooth (and “desaccaded”) eye velocity trace across trials for each direction-bias condition for a representative participant (s6) who experienced all direction-bias conditions.

Figure 2 reveals a gradual tendency to use predictive information about future motion as the smooth velocity traces show increasing anticipatory behavior when the uncertainty about the direction of future motion decreases. These results nicely replicate previous findings obtained from our and other groups under similar experimental conditions (Montagnini et al., 2010; Santos & Kowler, 2017). Figure 3a shows the distribution of anticipatory eye velocity for participant s6 highlighting the gradual shift of the whole distribu-
<figure><img src="image_8.png" /><figcaption>Figure 4. Boxplots of anticipatory eye velocity for the whole group of participants of Experiment 1 in all direction-bias sessions. Data are grouped depending on the target direction in the preceding trial. Individual mean velocities and 95% confidence interval are also presented with black symbols connected by thin lines. The individual data points' abscissae are randomly jittered on the x-axis in order to avoid the superposition of the symbols.</figcaption></figure>

tions across direction-bias sessions. We compared the mean velocities recorded during the anticipation window by means of individual one-way ANOVAs (with the probability bias as three-level single factor) and post hoc all pair-wise comparisons using Tukey's HSD test. Boxplots in Figure 3b illustrate the results at the group level (with interquartile range 0.25-0.75 and outliers marked as a +) summarizing the anticipatory velocity distributions for each session. We also plotted the individual mean anticipatory velocity across sessions for each participant. The general tendency observed for participant s6, i.e., the increase of anticipatory velocities with the increase of direction probability, holds for all subjects. We found a systematic significant effect of the direction bias (*p* < 0.05) with large size effect (given by Cohen's *d* statistic), except for participant s11.

Previous work (Heinen et al., 2005; Kao & Morrow, 1994; Kowler et al., 1984; Kowler & McKee, 1987) has highlighted the role of local sequential effect upon anticipatory smooth pursuit. One could speculate, for instance, that the simplest mechanism underlying the observed effect of direction-bias could be that anticipatory eye velocity is determined by the target direction in the previous trial. In the *P* = 0.9 bias condition, for instance, there are many more trials preceded by a rightward target motion than by a leftward one. Hence, if a rightward trial is systematically followed by rightward anticipatory pursuit (and the other way around for leftward trials), anticipation could be, on average, strongly biased, in the *P* = 0.9 bias condition,

in favor of rightward velocity. A first observation that makes this possibility unlikely is that the distributions of anticipatory velocity (see Figure 3a for a representative participant) look consistently unimodal for all direction-bias conditions, whereas a bimodal shape would have been expected (with increasingly uneven peaks for increasing bias) if the previous trial's direction had completely determined the anticipatory pursuit velocity. Nonetheless, we tested whether the direction of target motion of the immediately preceding trial could at least partly explain the observed bias in anticipatory velocity by comparing the latter for trials pooled depending on the previous trial direction. Figure 4 shows the group boxplots for a SPEM velocity together with the individual data (thin black lines and symbols): An effect of the preceding trial on a SPEM appears, indeed, when the direction bias is above 0.5. However, the difference in anticipatory velocity due to the previous trial's direction (illustrated by the difference between light and dark blue boxes) is consistently smaller than the probability-bias effect across experimental blocks. A two-way repeated-measures ANOVA with previous trial direction (right/left) and direction bias (*P* = 0.5, 0.75, and 0.9) as factors corroborated this observation, indicating that both main factors and their interaction affect significantly the anticipatory velocity (*p* < 0.05). We come back to a systematic, model-based analysis of the trial sequence effects later in this manuscript.

We examined the main parameters of visually guided smooth pursuit (visual pursuit latency, pursuit acceleration, steady-state velocity) and did not find any systematic effect of the direction-bias manipulation. The rate of visually guided saccades during target tracking did not systematically change across direction-bias conditions either. In contrast, the latency of the first visually guided saccade increased significantly with the direction bias. This finding may be explained by the fact that when the target motion direction is coherent with the direction of anticipatory smooth pursuit, the retinal slip at the initiation of visually guided pursuit is reduced, and so is the retinal spatial distance between the moving target and the fovea. The reduction of retinal velocity and position error is more pronounced when pursuit anticipation is stronger, and this may naturally increase the delay for the execution of the first catch-up saccade.

## Experiment 2: Operant conditioning paradigm

This experiment aimed at probing the effects of reward contingencies on anticipatory eye movement velocity. Figure 5 illustrates the mean ± 95% confidence interval of anticipatory eye velocity for each direction-bias session and the corresponding rein-
<figure><img src="image_9.png" /><figcaption>Figure 5. Mean and 95% confidence interval of the mean anticipatory smooth eye velocity in each direction-bias session and reinforcement condition of Experiment 1 for participant s6.</figcaption></figure>

reinforcement condition for participant s6. Much as in Experiment 1, anticipatory eye velocities increased with the motion-direction probability. However, it appears that these changes also depend on the reward condition, in particular with the *P* = 0.9 and *P* = 1 direction bias.

Even though the direction-bias effect seems to be
dominant and the reinforcement effects are strongly
variable across direction-bias conditions, a SPEM is
generally larger in the booster versus the braker
condition as summarized in Figure 6. We performed a
one-way ANOVA analysis to test for statistical
differences between reinforcement conditions for each
direction-bias condition and for each individual par-
ticipant. Significant differences were found in the *P* =
0.50 direction bias for all participants (*p* < 0.05). Post
hoc, all pair-wise Tukey HSD analysis revealed that
anticipatory velocities were significantly higher in the
booster condition than in baseline condition for all
participants (*p* < 0.05). However, and surprisingly, for
participants s6, s16, and s17, the mean anticipatory eye
velocity in the braker condition was significantly higher
than in the baseline condition (*p* < 0.05). In the *P* =
0.75 direction-bias condition, no significant differences
were found between reinforcement conditions for
participant s6 (*F* = 0.307, *p* = NS), s17 (*F* = 2.393, *p* =
NS), and s18 (*F* = 2.964, *p* = NS). Significant differences
were found for participant s5 (*F* = 13.939, *p* < 0.05), s7
(*F* = 23.144, *p* < 0.05), s8 (*F* = 17.435, *p* < 0.05), and
s16 (*F* = 8.328, *p* < 0.05). Post hoc, all pair-wise Tukey
HSD analysis revealed higher anticipatory mean
velocities in the booster compared to both the baseline
and braker conditions for participants s5, s7, and s8 (*p*
< 0.05) and braker anticipatory mean velocities
significantly below baseline and booster velocity for
participant s16 (*p* < 0.05). The other pair-wise

<figure><img src="image_10.png" /><figcaption>Figure 6. Boxplots of anticipatory eye velocity for the whole group of participants in all direction-bias sessions and reinforcement conditions of Experiment 1. Individual mean velocities and 95% confidence interval are also presented with black symbols connected by thin lines. The individual data points' abscissae are randomly jittered on the x-axis in order to avoid the superposition of the symbols.</figcaption></figure>

comparisons were not significant. Finally, for *P* = 0.90
direction bias, we found statistically significant differ-
ences for all participants. Post hoc, both all pair-wise
Tukey HSD and 95% confidence interval analyses
showed anticipatory mean velocities systematically
below baseline velocities in the braker condition (*p* <
0.05). We did not find significant differences between
booster and baseline velocities for participants s8, s16,
s17, and s18, but an effect of the booster condition was
found for participants s5, s6, and s7. It is important to
note, however, that, contrary to the direction-bias
effect, the effect size values for the operant-condition-
ing modulation (given by the eta square value) would
conventionally be considered as within a small size
range.

The parameters of visually guided smooth pursuit
did not differ significantly across reinforcement condi-
tions. For instance, the rate of visually guided catch-up
saccades did not vary systematically between the
baseline and the booster or braker sessions. The latency
of the initial catch-up saccade, in contrast, increased
slightly in the booster (mean increase by 15 ms) and
even more in the braker condition (mean increase by 40
ms) with respect to baseline.

To further probe the effects of reward contingencies
on smooth eye movement velocity during the antici-
pation window and rule out the possibility that those
effects would be instead related to unspecific reward
delivery, we implemented a yoked-control procedure in
which rewards and punishments were not contingent on
anticipatory smooth pursuit. If the operant nature of
the anticipatory behavior is critical, then we would
<figure><img src="image_11.png" /><figcaption>Figure 7. Mean and 95% confidence interval of the mean anticipatory smooth eye velocity in each direction-bias session for the two participants (s8 and s18) that experienced all reinforcement and yoked conditions.</figcaption></figure>

expect anticipatory velocity in the yoked group to be reduced with respect to the booster condition and enhanced with respect to the braker condition at least for the *P* = 0.90 direction bias. In other words, only small differences are expected between the data of the yoked-control group (in which reward was delivered randomly across trials) and the baseline condition. Figure 7 shows the mean ± 95% confidence interval of anticipatory eye velocity for each direction-bias session in all reinforcement conditions for participants s8 and s18 who participated in all conditions of Experiment 2.

Overall, we observe a decrease of aSPEM in the yoked condition with respect to the booster condition for all direction-bias sessions for both s8 and s18. For participant S8, the yoked anticipatory velocity was also reduced with respect to the baseline condition for all direction biases (*P* = 0.50: *T* = -4.953, *p* < 0.05; *P* = 0.75: *T* = -4.042, *p* < 0.05; *P* = 0.90: *T* = -11.646, *p* < 0.05). Participant s18 shows a significant reduction for *P* = 0.75 (*T* = -3.975, *p* < 0.05) but not for *P* = 0.50 (*T* = 0.893, *p* = NS) and *P* = 0.90 (*T* = 0.525, *p* = NS). Figure 8 allows us to compare anticipatory eye velocity in the yoked condition with the baseline condition for each of the four participants of the yoked procedure, highlighting a general mild reduction in aSPEM with some variability across participants.

In summary, the results of the yoked-control procedure suggest that the random delivery of reward does not increase anticipatory eye velocity: aSPEM velocities were systematically lower compared to the velocities in the booster condition. On the other hand, the present results suggest also an unexpected generic "attenuating" effect, slightly reducing anticipatory velocity compared to the baseline condition. Interestingly, somewhat similar findings of a possible negative effect of random delivery of reward have been reported

<figure><img src="image_12.png" /><figcaption>Figure 8. Boxplots of anticipatory eye velocity in the baseline and yoked-control condition for the group of yoked participants across all direction-bias sessions. Individual mean velocities and 95% confidence interval are also presented with black symbols connected by thin lines. The individual data points' abscissae are randomly jittered on the x-axis in order to avoid the superposition of the symbols.</figcaption></figure>

in the learning literature (Thompson & Iwata, 2005; Twining, Bolan, & Grigson, 2009); we come back to this point in the discussion.

## Experiment 3: Associative learning

This experiment was designed to probe the effects of a response-independent reinforcement, whereby the reward is simply associated to the stimulus condition and independent from the oculomotor behavior. In other words, we test the hypothesis that anticipatory smooth eye velocity is modulated by the relative expectation to be rewarded when the target moves in a specific direction. Here, we had three sessions with equal probability of rightward and leftward target motion direction (*P* = 0.50) but different reward probabilities. One might expect to observe a proportional increase of anticipatory smooth eye velocity in a given direction as a function of the probability to obtain a reward when the target moves in that direction. The higher the directional bias for reward likelihood, the higher the expected anticipatory velocity. This effect has been observed only in three participants out of six but to different extents. Participant s19 had aSPEM velocities significantly above the baseline (*P* = 0.5) velocities in all conditions *P*<sub>rew</sub> = 0.50 (*T* = 5.965, *p* < 0.05), *P*<sub>rew</sub> = 0.75 (*T* = 6.147, *p* < 0.05), *P*<sub>rew</sub> = 0.90 (*T* = 4.872, *p* < 0.05). The other two participants had a significant difference only in the *P*<sub>rew</sub> = 0.90 condition (s5, *T* = 3.704, *p* < 0.05; s15, *T* =
<figure><img src="image_13.png" /><figcaption>Figure 9. Boxplots of anticipatory eye velocity in the baseline and associative learning condition for the whole group of participants of Experiment 3 as a function of the probability of reward for rightward trials *P*<sub>rew</sub>. Individual mean velocities and 95% confidence interval are also presented with black symbols connected by thin lines. The individual data points' abscissae are randomly jittered on the x-axis in order to avoid the superposition of the symbols.</figcaption></figure>

2.353, *p* < 0.05). For the three other participants, no significant difference from the baseline condition was detected. These results suggest that the reward expectancy, per se, can indeed shape the anticipatory smooth pursuit in some participants but with a weak effect and stronger interindividual differences than operant conditioning. Figure 9 summarizes all results of Experiment 3 by means of boxplots of the mean anticipatory velocity recorded for all participants in both conditions.

Discussion

Our data has shown that subjects are able to use
statistical regularities of motion features to produce
consistent aSPEMs. We found a monotonic relation
between the mean aSPEM velocity and the bias applied
to motion direction. This robust effect was found in all
participants in the baseline experiment with no
exception. We probed the operant nature of this
behavior by implementing an eye velocity–contingent
paradigm in which rewards depended on aSPEM
velocities. We observed an influence of the reward
contingencies on aSPEM, albeit with smaller and less
robust effects than the direction-bias, per se. We further
probed the operant nature of aSPEM by implementing
a yoked-control condition, delivering reward randomly
with no systematic association to oculomotor behavior.
This manipulation resulted in a slight decrease of

aSPEM velocities. Finally, we implemented an asso-
ciative learning paradigm by systematically associating
a specific target direction to reward probability. Our
data showed a much smaller effect and greater
intersubject variability than in the operant paradigm.
Overall, these results lead us to conclude that, although
sensorimotor probabilistic information about target
motion seems to be dominant in controlling aSPEM
during the course of our experiments, anticipatory
pursuit can also be modulated by reward, especially
within an operant-conditioning framework.

Integration of statistical regularities: The
direction bias

Throughout the first experimental condition, we
investigated the integration of environmental statistical
regularities in a smooth tracking task with a family of
direction biases for the target motion. We found a
global and robust effect of direction bias on anticipa-
tory smooth pursuit. These results are coherent with
previous oculomotor findings by our and also other
groups (Montagnini et al., 2010; Santos & Kowler,
2017; Souto, Montagnini, & Masson, 2008). Typically,
aSPEM is observed after a temporal cue and ahead of
target motion onset (Kowler & Steinman, 1979a,
1979b; Kowler et al, 1984), and it depends on the
previous trials experience whether or not the latter
provides statistically reliable information (Heinen et al.,
2005; Kao & Morrow, 1994; Kowler & McKee, 1987).
Importantly, even in highly predictable situations, the
occurrence of fixation trials interleaved with tracking
trials can perturb pursuit anticipation (Watamaniuk,
Bal, & Heinen, 2017). Some classical experiments have
demonstrated the existence of prediction-based smooth
pursuit during the transient disappearance of a moving
target (Becker & Fuchs, 1985). Overall, it is now clear
that smooth pursuit behavior can be modulated even in
the absence of online sensory stimulation. We have
chosen to study aSPEMs in order to probe the
sensitivity of expectancy-based oculomotor behavior to
global (i.e., on a long time scale, several hundreds of
trials) and local (on a short time scale, few trials)
regularities and to investigate how sensorimotor
expectancy interacts with reward contingencies in
shaping oculomotor behavior without direct sensory
feedback. In previous studies (Montagnini et al., 2010;
Souto et al., 2008), we have analyzed how forthcoming
motion properties, such as target speed or direction,
can be predicted and anticipated with coherent
orienting eye movements. We found a graded effect of
both the speed and the direction bias with mean
anticipatory eye velocity linearly related to the prob-
ability of motion's speed or direction. Here, we
replicated part of those results, using a limited number
of direction probability biases and strengthened them by generalizing them on a large pool of participants. These results imply that the probability bias over a target's direction is one additional factor beyond other physical and cognitive cues (Kowler et al., 2014; Santos & Kowler, 2017) that modulate the common predictive framework driving anticipatory behavior to optimize a rapid and precise foveation of the target on its most expected future path.

## Direction bias and reward contingencies

The plasticity of pursuit eye movements during tracking of a visual target has previously been studied, on both humans (K. Fukushima, Tanaka, Suzuki, Fukushima, & Yoshida, 1996; Ogawa & Fujita, 1997; Optican, Zee, & Chu, 1985; Takagi et al., 2000; Takagi, Trillenberg, & Zee, 2001) and monkeys (Bourrelly, Quinet, Cavanagh, & Goffart, 2016; Kahlon & Lisberger, 1996). Here, we wanted to investigate the reinforcement-based plasticity of predictive smooth pursuit. In particular, with Experiment 2, we aimed at strengthening and expanding an already established result by Madelain and Krauzlis (2003a) to the case of expectancy-based aSPEM. Madelain and Krauzlis (2003a) have established that the natural smooth pursuit deceleration observed during the transient disappearance of the tracked target could be considerably reduced by reinforcement contingencies in the absence of any visual feedback. Overall, we found differences between individual aSPEM velocities in operant-conditioning sessions compared to the corresponding baseline sessions. However, those differences become mostly significant with a *P* ≥ 0.90 direction-bias session, i.e., for the highest level of predictability we tested. Importantly, and contrary to our expectation, the anticipatory velocity was sometimes larger in the *braker* condition than in the baseline when *P* was lower than 0.90 (e.g., for participants s6, s16, and s17). One possible explanation of this can simply be an effect of training carried over across successive sessions. Indeed, all participants experienced the baseline condition prior to the other experimental conditions. This might have trained participants and increased their sensitivity to direction biases or modified their prior expectation of a target going rightward. Results obtained for the *P* = 0.90 direction-bias session confirmed our hypothesis regarding the effect of reinforcement on anticipation as all participants had lower aSPEM velocities in the braker condition compared to baseline, strengthening the notion that a potential task learning-related boost of aSPEM is not always sufficient to counteract the reduction of aSPEM due to reinforcement contingencies. Concerning the booster conditions, we generally found an increase of

anticipatory velocity with respect to the baseline. However, such increase was significant only for participants s5, s6, and s7 in the *P* = 0.9 condition. Several factors may explain the lack of a more robust positive effect of reward contingencies in the booster condition for the strongest direction bias: First, the adaptive criterion for reinforcement was constantly evolving toward more demanding levels, such that, at some point, the required aSPEM velocity to obtain a reward became extremely high, plausibly close to the maximum of the anticipatory velocity range. Second, this increase in task demand might also have affected our participants' attention and motivation. Third, because trials with saccades in the critical comparison window were punished, at some point participants had presumably to actively inhibit excessively strong anticipation to avoid these saccades. When we looked at the rate of trials with anticipatory saccades across participants, we found indeed a larger rate in the baseline condition than in the booster condition with about 18% less anticipatory saccade trials in the latter. If this decrease suggests that the punishment criterion for saccades has been taken into account, participants might have had to cope with somehow conflicting aims: the need to anticipate accurately to provide fast foveation and get rewarded and also the need to keep anticipatory velocity below the threshold for saccade detection and punishment.

Nonetheless, our results demonstrate overall that aSPEMs are sensitive to the rewarding value of a stimulus even when the reward is related to the nonvisually guided phase of pursuit rather than the visually guided phase (Brielmann & Spering, 2015; Joshua & Lisberger, 2012; Schütz et al., 2015) and that reinforcement contingencies have to be taken into account in addition to attentional factors (Khurana & Kowler, 1987) and prediction (G. R. Barnes, Barnes, & Chakraborti, 2000; G. R. Barnes & Donelan, 1999; Santos & Kowler, 2017; Wells & Barnes, 1998) for the buildup of prior expectation about a target's direction.

## Integration of environmental regularities across trials: A model-based analysis

Integrating information about the regularities in our environment is essential to build a general knowledge of it whether the nature of that information is statistical or reward-related. To accurately build up this integration, one must rely on the previous trials, but one might prioritize recent information with respect to very remote events. In a study on the sequential effects of target speed, Maus, Potapchuk, Watamaniuk, and Heinen (2015) proposed a simple model in which anticipatory smooth pursuit eye velocity is a linear function of the mean target speed across the recent trial
<figure><img src="image_14.png" /></figure>

Figure 10. Scatterplots of mean anticipatory velocity (data pooled from all subjects) and linear regression lines between anticipatory velocities and the estimated direction bias over the 10 past trials for the different direction-bias sessions. For an easier visualization, data corresponding to different direction-bias sessions are slightly shifted with respect to each other. Within-condition data points are further jittered on the x-axis in order to avoid massive superposition of points.

history. In a similar way, we developed a model-based analysis that aims at elucidating the relation between anticipatory smooth pursuit eye velocity and local estimate of the direction bias. As we can see in Figure 10 (in which we pooled all our participants' data together), for an arbitrary 10-trial memory, there is indeed a noisy but significant linear relation between the estimated probability of rightward direction (defined as the mean of a binary function associating one to rightward trials and zero to leftward trials across the previous 10 trials) and the recorded anticipatory velocity. The analysis of the linear regression for all individual data pooled together and a memory size of 10 trials yields $P = 0.50: R = 0.13, p < 0.05; P = 0.75: R = 0.21, p < 0.05$; and $P = 0.90: R = 0.24, p < 0.05$, where R is the Pearson's correlation coefficient.

Similar to Maus et al. (2015), we repeated this analysis for several memory sizes to test how Pearson's correlation coefficient varies when taking into account longer chunks of the recent history. With our analysis of the trial-sequence effects, we also wanted to address the complementary question of whether the weight of recent trials is affected by the global direction probability or by the reinforcement contingency. One possibility is, for instance, that the association between reward and anticipatory behavior potentiates the effect of the trial sequence. We, thus, analyzed the linear regression across different direction-bias and reward conditions. As we can see in Figure 11, we replicated, with lower *R* however, the general tendency found by Maus et al. (2015) of a higher Pearson's *R* for smaller memory sizes. A very similar dependence of the correlation coefficient upon tau was observed across all direction-bias and reinforcement conditions. For memory size below 20, the direction bias does also

systematically affect the Pearson's *R* with higher *R* for the stronger direction bias.

Altogether, anticipatory eye velocity seems to be weakly correlated to the local estimate of the target direction bias, which relies on a short-term memory integration (~10 trials). Further work and larger statistical power would be needed to address the question of whether reinforcement contingencies interact with this relation. At this stage, this linear regression explains only a small fraction of the variability in our data, and this analysis does not allow any firm quantitative conclusion. A more advanced dynamical model for the integration of statistical regularities in target motion direction might be better suited to explain the experimental data.

## Reward unpredictability

To further confirm that the effect observed in Experiment 2 was truly due to reinforcement contingencies we used the well-established yoked-control procedure (Wasserman, 2010). To our knowledge, the study by Madelain and Krauzlis (2003a) is the only one that reported a yoked-control procedure in a smooth pursuit task with reinforcement. Similar to Madelain and Krauzlis, the yoked condition did not result in any enhancement of anticipatory smooth pursuit velocity compared to the baseline condition, and the measured anticipatory velocity was reduced compared to the booster condition of the operant-conditioning procedure. However, we would have expected the distribution of aSPEM velocities in the yoked group to be statistically similar to the one in the baseline (like for
<figure><img src="image_15.png" /><figcaption>Figure 11. Pearson's correlation coefficient *R* for the linear regression of anticipatory velocity (group-pooled data) and the local estimation of direction bias across all direction-bias and reinforcement conditions and for different memory-storage sizes (Tau).</figcaption></figure>

the Madelain & Krauzlis study), but on three out of four participants, we obtained yoked-control mean anticipatory velocity slightly below the baseline.

Although unexpected in the context of our study, similar counterintuitive results have been reported in the literature in other contexts. Some examples of the effects of delivering rewards independently from the organism's behavior can be found in the so-called *superstitious experiments* (Skinner, 1948): When putting food-deprived pigeons in mere food delivery situations (a 5-s access to food every 15 s), they may exhibit superstitious behavior, e.g., repeating specific responses (wings spreading, head shifting, etc.) that had been temporally correlated with food delivery as the training progressed. In a study with human participants, Ono (1987) used a single response independent schedule of reinforcement with variable times. He found that random reinforcers influenced human behavior and, in some cases, observed stable behavioral patterns. In summary, reinforcers in yoked-control conditions may influence the participant's behavior. Although this remains a speculative explanation, the variability of observed behaviors when reward delivery is independent from the response seems to be a plausible ground for the variability of the effects of our yoked manipulation.

## Stimulus-associated reward expectancy

To separate the effects of the association between target direction and reward from those produced by operant conditioning, we systematically delivered rewards in different proportions across the two target motion directions in the absence of a contingency

between the motor action and the reward occurrence. This response-independent reinforcement paradigm addresses the effects of reward delivery associated to a particular stimulus condition separately from those of the response contingency. What we observed in our associative learning experiment is a higher variability and an overall smaller size of the effect on aSPEM than in an operant-conditioning paradigm. Interestingly, and similar to the present study, in a recent article on reward-cued saccadic eye movements, different and uncorrelated effects have been reported for response-contingent and noncontingent reinforcement (Manohar, Finzi, Drew, & Husain, 2017). Several factors can have influenced these results. In our operant paradigm, a punishment or a reward is delivered on each trial, depending on the aSPEM velocity. In our associative-learning paradigm, a reinforcer is given from time to time independently from the oculomotor response. This form of learning depends on a low-level substitution of one unconditioned stimulus that triggers the response by a conditioned one (Pavlov, 1927). Given the results we obtained in the operant and yoked procedure, one interpretation could be that aSPEMs rely on a higher cognitive process rather than a low-level associative process. Another explanation might be related to the absence of punishment as only rewards were given in the associative-learning experiment. Participants always finished the experiment with the highest amount of cumulative reward they can get regardless of their behavior, and this could have reduced their motivation according again to the Yerkes-Dodson law (Yerkes & Dodson, 1908) stating that a not enough challenging task will be passively executed. Although our results on the associative-learning paradigm cannot allow us to fully disentangle the operant and Pavlovian processes
involved in a SPEM, they, however, suggest the
dominance of operant components.

Neural bases of contextual contingencies
modulation in smooth eye movements

Extensive research has addressed, in the past, the
neuronal bases of reward modulations of motor (and
oculomotor) control. The basal ganglia have been
described as specifically related to the effects of reward
on general behavior (Joshua, Adler, Mitelman, Vaadia,
& Bergman, 2008; Lauwereyns, Watanabe, Coe, &
Hikosaka, 2002; Schultz, 1998) and on eye movements
in particular (Hikosaka, 2007; Yoshida & Tanaka,
2009). Importantly, the basal ganglia have strong
anatomical connections with the frontal eye fields
(FEFs), a major visuo-oculomotor area in the pre-
frontal cortex and, in particular, with the subregion of
the FEF that is selectively involved in smooth eye
movement (FEFsem; Cui, Yan, & Lynch, 2003; Tian &
Lynch, 1997). In addition, one of the major outputs of
the basal ganglia is the superior colliculus, which is
known for its critical role for saccadic control and
which has been recently shown to be involved in
smooth pursuit initiation and target selection (Num-
mela & Krauzlis, 2010, 2011). Thus, the cortico-
subcortical network, including the FEF, the superior
colliculus, and the basal ganglia (and possibly other
regions not discussed here), is probably a good
candidate as a major substrate of the reward-based
modulations of smooth eye movements observed here
and in previous studies.

The neuronal pathways involved in anticipatory eye
movements have also been studied, in both monkeys
and humans. In nonhuman primates, the correct
execution of cued, expectancy-based pursuit is related
to selective neuronal activity in the supplementary eye
fields (SEFs) in the dorso-medial frontal cortex (de
Hemptinne et al., 2008; Missal & Heinen, 2004) as well
as in the FEFsem (Keating, 1993; Raghavan & Joshua,
2017). Different neurons in the FEFsem seem to
contribute to different phases of pursuit and learning in
pursuit processes (Li & Lisberger, 2011; Schoppik,
Nagel, & Lisberger, 2008), and local modulations of
neuronal activity related to pursuit learning of
instructed motion properties have been also reported in
the cerebellum (Li, Medina, Frank, & Lisberger, 2011;
Yang & Lisberger, 2010). The FEF is also involved in
processing extraretinal pursuit components, during
both transient target-blank tasks (K. Fukushima, Sato,
Fukushima, Shinmei, & Kaneko, 2000; K. Fukushima,
Yamanobe, Shinmei, & Fukushima, 2002; Tanaka &
Fukushima, 1998) and memory-based pursuit (J.
Fukushima et al., 2011). In humans, Gagnon, Paus,
Grosbras, Pike, and O'Driscoll (2006) applied trans-

cranial magnetic stimulation (TMS) to the FEFsem
and SEF during sinusoidal pursuit to assess its effects
on the pursuit response to predictable changes in target
motion rather than unexpected ones. For the FEFsem,
they found that TMS applied immediately before the
target reversed direction increased eye velocity in the
new direction, whereas TMS applied in midcycle
(immediately before the target began to slow down),
decreased eye velocity. For the SEF, TMS applied at
target reversal increased eye velocity in the new
direction but had no effect on eye velocity when applied
at midcycle.

In conclusion, understanding the neuronal bases of
the anticipatory smooth eye movements is still an open
challenge, and the important issue that is addressed by
recent studies is to identify the internal signals that
have access to anticipatory control (Bosco et al., 2015;
Wolpert & Landy, 2012; Zhao & Warren, 2015),
including reward signals.

Conclusion

When moving stimuli are sequentially presented
according to direction bias, observers adapt to the
current bias by producing a SPEM in a range of velocity
that helps in minimizing the foveation delay. When we
add eye velocity-contingent reward beyond the senso-
rimotor contingencies, we manage to slightly but
significantly modulate a SPEM following the eye
velocity-contingent punishment/reward schedule in
force. In case of inconsistency between oculomotor
behavior and reward, a SPEM behavior undergoes a
little reduction. Finally, an associative learning para-
digm showed smaller effects and larger intersubjects
variability than the operant conditioning. Taken
together, these results suggest that a SPEM are a flexible
behavior, relying primarily on perceptual and statistical
cues, but are also modulated by reward-contingencies;
smooth pursuit might be optimized for action conse-
quences rather than passive delivery of reward.

Keywords: smooth pursuit, prediction, anticipation,
operant conditioning, probabilistic bias

Acknowledgments

This work was supported by Agence Nationale de
Recherche grant ANR-13-APPR-008 and EU Marie-
Sklodowska-Curie Grant No 642961. We thank
Jeremie Jozefowiez for commenting on an earlier
version of the manuscript.

Commercial relationships: none.
Corresponding author: Anna Montagnini.

E-mail: anna.montagnini@univ-amu.fr.

Address: Aix Marseille Université, CNRS, Institut de Neurosciences de la Timone UMR 7289, Marseille, France.

## References

Badler, J. B., & Heinen, S. J. (2006). Anticipatory movement timing using prediction and external cues. *The Journal of Neuroscience*, 26, 4519–4525.

Barnes, G., Grealy, M., & Collins, S. (1997). Volitional control of anticipatory ocular smooth pursuit after viewing, but not pursuing, a moving target: Evidence for a re-afferent velocity store. *Experimental Brain Research*, 116(3), 445–455.

Barnes, G. R. (2008). Cognitive processes involved in smooth pursuit eye movements. *Brain & Cognition*, 68, 309–326.

Barnes, G. R., & Asselman, P. T. (1991). The mechanism of prediction in human smooth pursuit eye movements. *The Journal of Physiology*, 439, 439–461.

Barnes, G. R., Barnes, D. M., & Chakraborti, S. R. (2000). Ocular pursuit responses to repeated, single-cycle sinusoids reveal behavior compatible with predictive pursuit. *Journal of Neurophysiology*, 84, 2340–2355.

Barnes, G. R., & Donelan, S. F. (1999). The remembered pursuit task: Evidence for segregation of timing and velocity storage in predictive oculomotor control. *Experimental Brain Research*, 129, 57–67.

Becker, W., & Fuchs, A. F. (1985). Prediction in the oculomotor system: Smooth pursuit during transient disappearance of a visual target. *Experimental Brain Research*, 57, 562–575.

Bennett, S. J., & Barnes, G. R. (2006). Combined smooth and saccadic ocular pursuit during the transient occlusion of a moving visual object. *Experimental Brain Research*, 168, 313–321.

Berman, R. A., Colby, C. L., Genovese, C. R., Voyvodic, J. T., Luna, B., Thulborn, K. R., & Sweeney, J. A. (1999). Cortical networks subserving pursuit and saccadic eye movements in humans: An FMRI study. *Human Brain Mapping*, 8(4), 209–225.

Beutter, B. R., & Stone, L. S. (2000). Motion coherence affects human perception and pursuit similarly. *Visual Neuroscience*, 17, 139–153.

Bogadhi, A. R., Montagnini, A., & Masson, G. S.

(2013). Dynamic interaction between retinal and extraretinal signals in motion integration for smooth pursuit. *Journal of Vision*, 13(13):5, 1–26, https://doi.org/10.1167/13.13.5. [PubMed] [Article]

Bosco, G., Delle Monache, S., Gravano, S., Indovina, I., La Scaleia, B., Maffei, V., ... Lacquaniti, F. (2015). Filling gaps in visual motion for target capture. *Frontiers in Integrative Neuroscience*, 9: 13.

Bourrelly, C., Quinet, J., Cavanagh, P., & Goffart, L. (2016). Learning the trajectory of a moving visual target and evolution of its tracking in the monkey. *Journal of Neurophysiology*, 116, 2739–2751.

Brainard, D. H. (1997). The Psychophysics Toolbox. *Spatial Vision*, 10, 433–436.

Brielmann, A. A., & Spering, M. (2015). Effects of reward on the accuracy and dynamics of smooth pursuit eye movements. *Journal of Experimental Psychology: Human Perception and Performance*, 41(4), 917–928.

Chen, Y., Holzman, P. S., & Nakayama, K. (2002). Visual and cognitive control of attention in smooth pursuit. *Progress in Brain Research*, 140, 255–265.

Collewijn, H., & Tamminga, E. P. (1984). Human smooth and saccadic eye movements during voluntary pursuit of different target motions on different backgrounds. *The Journal of Physiology*, 351, 217–250.

Cui, D. M., Yan, Y. J., & Lynch, J. C. (2003). Pursuit subregion of the frontal eye field projects to the caudate nucleus in monkeys. *Journal of Neurophysiology*, 89(5), 2678–2684.

Culham, J. C., Bran, S. A., Cavanagh, E., Kanwisher, N. G., Dale, A. M., & Tootell, R. B. (1998). Cortical fMRI activation produced by attentive tracking of moving targets. *Journal of Neurophysiology*, 80(5), 2657–2670.

Darcheville, J.-C., Madelain, L., Buquet, C., Charlier, J., & Miossec, Y. (1999). Operant conditioning of the visual smooth pursuit in young infants. *Behavioural Processes*, 46(2), 131–139.

de Hemptinne, C., Lefevre, P., & Missal, M. (2008). Neuronal bases of directional expectation and anticipatory pursuit. *Journal of Neuroscience*, 28, 4298–4310.

Ferrera, V., & Lisberger, S. G. (1995). Attention and target selection for smooth pursuit eye movements. *Journal of Neuroscience*, 15, 7472–7484.

Freyberg, S., & Ilg, U. J. (2008). Anticipatory smooth-pursuit eye movements in man and monkey. *Experimental Brain Research*, 186, 203–214.

Fukushima, J., Akao, T., Shichinohe, N., Kurkin, S., Kaneko, C. R., & Fukushima, K. (2011). Neuronal
activity in the caudal frontal eye fields of monkeys
during memory-based smooth pursuit eye move-
ments: Comparison with the supplementary eye
fields. *Cerebral Cortex*, 2 1(8), 1910–1924.

Fukushima, K., Sato, T., Fukushima, J., Shinmei, Y.,
& Kaneko, C. R. (2000). Activity of smooth
pursuit-related neurons in the monkey periarcuate
cortex during pursuit and passive whole-body
rotation. *Journal of Neurophysiology*, 83(1), 563–
587.

Fukushima, K., Tanaka, M., Suzuki, Y., Fukushima,
J., & Yoshida, T. (1996). Adaptive changes in
human smooth pursuit eye movement. *Neurosci-
ence Research*, 25, 391–398.

Fukushima, K., Yamanobe, T., Shinmei Y., & Fu-
kushima J. (2002). Predictive responses of periarc-
uate pursuit neurons to visual target motion.
*Experimental Brain Research, 145*, 104–120.

Gagnon, D., Paus, T., Grosbras, M.-H., Pike, G. B., & O'Driscoll, G. A. (2006). Transcranial magnetic stimulation of frontal oculomotor regions during smooth pursuit. *Journal of Neuroscience*, 26(2), 458-466.

Heinen, S. J., Badler, J. B., & Ting, W. (2005). Timing and velocity randomization similarly affect anticipatory pursuit. *Journal of Vision*, 5(6):1, 493–503, https://doi.org/10.1167/5.6.1. [PubMed] [Article]

Hikosaka, O. (2007). Basal ganglia mechanisms of reward-oriented eye movement. *Annals of the New York Academy of Sciences*, 1104, 229–249.

Ilg, U. J., & Churan, J. (2004). Motion perception without explicit activity in areas MT and MST. *Journal of Neurophysiology*, 92(3), 1512–1523.

Joshua, M., Adler, A., Mitelman, R., Vaadia, E., & Bergman, H. (2008). Midbrain dopaminergic neurons and striatal cholinergic interneurons encode the difference between reward and aversive events at different epochs of probabilistic classical conditioning trials. *The Journal of Neuroscience*, 28, 11673-11684.

Joshua, M., & Lisberger, S. G. (2012). Reward action in the initiation of smooth pursuit eye movements. *Journal of Neuroscience*, 32(8), 2856-2867.

Kahlon, M., & Lisberger, S. G. (1996). Coordinate system for learning in the smooth pursuit eye movements of monkeys. *The Journal of Neuroscience*, 16, 7270-7283.

Kao, G. W., & Morrow, M. J. (1994). The relationship of anticipatory smooth eye movement to smooth pursuit initiation. *Vision Research*, 34, 3027–3036.

Keating, G. E. (1993). Lesions of the frontal eye field impair pursuit eye movements, but preserve the

predictions driving them. *Behavioural Brain Research*, 53(1-2), 91–104.

Keller, E., & Khan, E. (1986). Smooth-pursuit initiation in the presence of a textured background in monkey. *Vision Research*, 36, 943-955.

Khurana, B., & Kowler, E. (1987). Shared attentional control of smooth eye movement and perception. *Vision Research*, 27, 1603-1618.

Kowler, E. (1989). Cognitive expectations, not habits, control anticipatory smooth oculomotor pursuit. *Vision Research*, 29, 1049-1057.

Kowler, E., Aitkin, C. D., Ross, N. M., Santos, E. M., & Zhao, M. (2014). Davida Teller Award Lecture 2013: The importance of prediction and anticipation in the control of smooth pursuit eye movements. *Journal of Vision*, 14(5):10, 1–16, https://doi.org/10.1167/14.5.10. [PubMed] [Article]

Kowler, E., Martins, A. J., & Pavel, M. (1984). The effect of expectations on slow oculomotor control-IV. Anticipatory smooth eye movements depend on prior target motions. *Vision Research*, 24(3), 197-210.

Kowler, E., & McKee, S. (1987). Sensitivity of smooth eye movement to small differences in target velocity. *Vision Research*, 27, 993-1015.

Kowler, E., & Steinman, R. M. (1979a). The effect of expectation on slow oculomotor control. I. Periodic target steps. *Vision Research*, 19(6), 619–632.

Kowler, E., & Steinman, R. M. (1979b). The effect of expectation on slow oculomotor control. II. Single target displacements. *Vision Research*, 19(6), 633-646.

Krauzlis, R. J., & Miles, F. A. (1996). Initiation of saccades during fixation or pursuit: Evidence in humans for a single mechanism. *Journal of Neurophysiology*, 76, 4175-4179.

Landelle, C., Montagnini, A., Madelain, L., & Danion, F. (2016). Eye tracking a self-moved target with complex hand-target dynamics. *Journal of Neurophysiology*, 116(4), 1859–1870.

Lauwereyns, J., Watanabe, K., Coe, B., & Hikosaka, O. (2002, July 25). A neural correlate of response bias in monkey caudate nucleus. *Nature*, 418, 413–417.

Li, J. X., & Lisberger, S. G. (2011). Learned timing of motor behavior in the smooth eye movement region of the frontal eye fields. *Neuron*, 69, 159–169.

Li, J. X., Medina, J. F., Frank, L. M., & Lisberger, S. G. (2011). Acquisition of neural learning in cerebellum and cerebral cortex for smooth pursuit eye movements. *Journal of Neuroscience*, 31, 12716–12726.
Lisberger, S. G., Morris, E. J., & Tychsen, L. (1987).
Visual motion processing and sensory-motor integration for smooth pursuit eye movements. *Annual Review of Neuroscience, 10*, 97-129.

Lorenceau, J. (2012). Cursive writing with smooth pursuit eye movements. *Current Biology*, 22(16), 1506-1509.

Madelain, L., & Krauzlis, R. J. (2003a). Effects of learning on smooth pursuit during transient disappearance of a visual target. *Journal of Neurophysiology*, 90, 972-982.

Madelain, L., & Krauzlis, R. J. (2003b). Pursuit of the ineffable: Perceptual and motor reversals during the tracking of apparent motion. *Journal of Vision*, 3(11):1, 642–653, https://doi.org/10.1167/3.11.1. [PubMed] [Article]

Madelain, L., Paeye, C., & Darcheville, J. C. (2011).
Operant control of human eye movements. *Behavioral Processes*, 87(1), 142-148.

Manohar, S. G., Finzi, R. D., Drew, D., & Husain, M. (2017). Distinct motivational effects of contingent and noncontingent rewards. *Psychological Science*, 28(7), 1016–1026.

Maus, G. W., Potapchuk, E., Watamaniuk, S. N. J., & Heinen, S. J. (2015). Different time scales of motion integration for anticipatory smooth pursuit and perceptual adaptation. *Journal of Vision*, 15(2):16, 1–13, https://doi.org/10.1167/15.2.16. [PubMed] [Article]

Missal, M., & Heinen, S. J. (2004). Supplementary eye fields stimulation facilitates anticipatory pursuit. *Journal of Neurophysiology*, 92(2), 1257–1262.

Montagnini, A., Mamassian, P., Perrinet, L. U., Castet, E., & Masson, G. S. (2007). Bayesian modeling of dynamic motion integration. *Journal of Physiology (Paris)*, 101, 64–77.

Montagnini, A., Souto, D., & Masson, G. (2010).
Anticipatory eye-movements under uncertainty: A window onto the internal representation of a visuomotor prior. *Journal of Vision*, 10(7):554, https://doi.org/10.1167/10.7.554. [Abstract]

Nummela, S. U., & Krauzlis, R. J. (2010). Inactivation of primate superior colliculus biases target choice for smooth pursuit, saccades, and button press responses. *Journal of Neurophysiology*, 104, 1538–1548.

Nummela, S. U., & Krauzlis, R. J. (2011). Superior colliculus inactivation alters the weighted integration of visual stimuli. *The Journal of Neuroscience*, 31(22), 8059–8066.

Ogawa, T., & Fujita, M. (1997). Adaptive modifications of human postsaccadic pursuit eye move-

ments induced by a step-ramp-ramp paradigm.
*Experimental Brain Research*, 116, 83-96.

Ono, K. (1987). Superstitious behavior in humans. *Journal of the Experimental Analysis of Behavior*, 47(3), 261-271.

Optican, L. M., Zee, D. S., & Chu, F. C. (1985).
Adaptive response to ocular muscle weakness in human pursuit and saccadic eye movements.
*Journal of Neurophysiology*, 54, 110-122.

Orban de Xivry J-J., Coppe S., Blohm G. and Lefèvre P. (2013) Kalman filtering naturally accounts for visually guided and predictive smooth pursuit dynamics. *The Journal of Neuroscience*, 33(44), 17301–17313.

Orban de Xivry, J. J., Missal, M., & Lefèvre, P. (2008). A dynamic representation of target motion drives predictive smooth pursuit during target blanking. *Journal of Vision*, 8(15):6, 1–13, https://doi.org/10.1167/8.15.6. [PubMed] [Article]

Pavlov, I. P. (1927). *Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex*. Oxford, England: Oxford University Press.

Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. *Spatial Vision*, 10, 437–442.

Perrinet, L. U., Adams, R. A., & Friston, K. J. (2014).
Active inference, eye movements and oculomotor delays. *Biological Cybernetics*, 108(6), 777–801.

Raghavan, R. T., & Joshua, M. (2017). Dissecting patterns of preparatory activity in the frontal eye fields during pursuit target selection. *Journal of Neurophysiology*, 118(4), 2216–2231.

Robinson, D. A. (1986). The systems approach to the oculomotor system. *Vision Research*, 26, 91-99.

Santos, E. M., & Kowler, E. (2017). Anticipatory smooth pursuit eye movements evoked by probabilistic cues. *Journal of Vision*, 17(13):13, 1–16, https://doi.org/10.1167/17.13.13. [PubMed] [Article]

Schoppik, D., Nagel, K. I., & Lisberger, S. G. (2008).
Cortical mechanisms of smooth eye movements revealed by dynamic covariations of neural and behavioral responses. *Neuron*, 58(2), 248–260.

Schultz, W. (1998). Predictive reward signal of dopamine neurons. *Journal of Neurophysiology*, 80(1), 1–27.

Schütz, A. C., Lossin, F., & Gegenfurtner, K. R. (2015). Dynamic integration of information about salience and value for smooth pursuit eye movements. *Vision Research*, 113, 169–178.

Skinner, B. F. (1948). "Superstition" in the pigeon. *Journal of Experimental Psychology*, 38, 168-172.
Souto, D., & Kerzel, D. (2008). Dynamics of attention during the initiation of smooth pursuit eye movements. *Journal of Vision*, *8*(14):3, 1–16, https://doi.org/10.1167/8.14.3. [PubMed] [Article]

Souto, D., Montagnini, A., & Masson, G. (2008). Scaling of anticipatory smooth pursuit eye movements with target speed probability. *Journal of Vision*, *8*(6):665, https://doi.org/10.1167/8.6.665. [Abstract]

Spering, M., Kerzel, D., Braun, D. I., Hawken, M. J., & Gegenfurtner, K. R. (2005). Effects of contrast on smooth pursuit eye movements. *Journal of Vision*, *5*(5):6, 455–465, https://doi.org/10.1167/5.5.6. [PubMed] [Article]

Spering, M., & Montagnini, A. (2011). Do we track what we see? Common versus independent processing for motion perception and smooth pursuit eye movements: A review. *Vision Research*, *51*, 836–852.

Stone, L. S., Beutter, B. R., & Lorenceau, J. (2000). Visual motion integration for perception and pursuit. *Perception*, *29*, 771–787.

Takagi, M., Abe, H., Hasegawa, S., Usui, T., Hasebe, H., Miki, A., & Zee, D. S. (2000). Context-specific adaptation of pursuit initiation in humans. *Investigative Ophthalmology & Visual Science*, *41*, 3763–3769.

Takagi, M., Trillenberg, P., & Zee, D. S. (2001). Adaptive control of pursuit, vergence and eye torsion in humans: Basic and clinical implications. *Vision Research*, *41*, 3331–3344.

Tanaka, M., & Fukushima, K. (1998). Neuronal responses related to smooth pursuit eye movements in the periarcuate cortical area of monkeys. *Journal of Neurophysiology*, *80*(1), 28–47.

Thompson, R. H., & Iwata, B. A. (2005). A review of reinforcement control procedures. *Journal of Applied Behavior Analysis*, *38*, 257–278.

Tian, J. R., & Lynch, J. C. (1997). Subcortical input to the smooth and saccadic eye movement subregions of the frontal eye field in Cebus monkey. *Journal of Neuroscience*, *17*, 9233–9247.

Twining, R. C., Bolan, M., & Grigson, P. S. (2009). Yoked delivery of cocaine is aversive and protects against the motivation for drug in rats. *Behavioral Neuroscience*, *123*(4), 913–925.

Tychsen, L., & Lisberger, S. G. (1986). Visual motion processing for the initiation of smooth-pursuit eye movements in humans. *Journal of Neurophysiology*, *56*, 953–968.

Van den Berg, A. V. (1988). Human smooth pursuit during transient perturbations of predictable and unpredictable target movement. *Experimental Brain Research*, *72*, 95–108.

Wallace, J. M., Stone, L. S., & Masson, G. S. (2005). Object motion computation for the initiation of smooth pursuit eye movements in humans. *Journal of Neurophysiology*, *93*, 2279–2293.

Wasserman, E. A. (2010). Yoked control procedure. In A. Salkind (Ed.), *Encyclopedia of research design*, (pp. 1649–1651). Lawrence, KS: Sage. https://doi.org/10.4135/9781412961288.

Watamaniuk, S. N. J., Bal, J., & Heinen, S. J. (2017). A subconscious interaction between fixation and anticipatory pursuit. *Journal of Neuroscience*, *37*, 11424–11430.

Wells, S. G., & Barnes, G. R. (1998). Fast, anticipatory smooth-pursuit eye movements appear to depend on a short-term store. *Experimental Brain Research*, *120*, 129–133.

Wells, S. G., & Barnes, G. R. (1999). Predictive smooth pursuit eye movements during identification of moving acuity targets. *Vision Research*, *39*, 2767–2775.

Wolpert, D. M., & Landy, M. S. (2012). Motor control is decision-making. *Current Opinion in Neurobiology*, *22*(6), 996–1003.

Yang, Y., & Lisberger, S. G. (2010). Learning on multiple timescales in smooth pursuit eye movements. *Journal of Neurophysiology*, *104*(5), 2850–2862.

Yerkes, R. M., & Dodson, J. D. (1908). The relation of strength of stimulus to rapidity of habit-formation. *Journal of Comparative Neurology and Psychology*, *18*, 459–482.

Yoshida, A., & Tanaka, M. (2009). Neuronal activity in the primate globus pallidus during smooth pursuit eye movements. *NeuroReport*, *20*, 121–125.

Zhao, H., & Warren, W. H. (2015). On-line and model-based approaches to the visual control of action. *Vision Research*, *110*, 190–202.